Securing Tools and Agents — Excessive Agency and Sandboxing
Least-privilege tool design, confirmation for risky actions, sandboxed execution, and limiting what a compromised agent can do.
A Tool Call Is a Request, Not an Authorization
An agent combines model-generated decisions with tools such as search, ticket lookup, email drafting, or account changes. Every added tool expands the system’s capability and the possible impact of unexpected behavior. The model may select a tool or propose arguments, but the application and tool service must independently decide whether the current principal may perform that exact operation.
OWASP’s 2025 LLM06 Excessive Agency describes risk from excessive functionality, permissions, or autonomy. In practical terms, do not give a course-answering assistant an account-deletion tool simply because another feature might use it later. Start with read-only, narrow capabilities; add write access only for a clear product need and protect it with authorization, approval, and audit controls.
Give Each Task the Minimum Useful Power
Design tools around one bounded business operation rather than exposing a generic database, shell, or unrestricted HTTP client. Separate read from write permissions. Bind every call to the authenticated user or service identity, restrict records and fields, set limits, and define which actions require a person’s confirmation. Keep credentials short-lived and scoped to the specific integration where possible.
| Design choice | Safer pattern | Riskier pattern |
|---|---|---|
| Operation | get_order_status(order_id) | run_arbitrary_database_query(sql) |
| Identity | Server injects verified actor context | Model supplies user_id or role as an argument |
| Permission | Tool checks record ownership and role | Prompt says “only access authorized records” |
| Writes | Narrow action with explicit approval | Agent can perform any account mutation autonomously |
| Credentials | Scoped service identity for one operation | Shared administrator credential for all tools |
Tool descriptions should be accurate and examples should avoid exposing secrets. Avoid overly broad tool responses: return only fields needed for the task, with clear provenance and error states. The model should not receive a credential merely to decide which API to call.
Validate the Actor, Arguments and Target
The example below demonstrates a narrow tool that checks its caller and validates a requested delivery slot before returning a proposal. It uses a fictional in-memory record. The model is not allowed to supply the actor identity; production code would obtain it from a verified server-side session and perform the write through an authorized service after any required approval.
from dataclasses import dataclass
@dataclass(frozen=True)
class Actor:
user_id: str
role: str
@dataclass
class Order:
order_id: str
owner_id: str
delivery_slot: str
def propose_delivery_change(actor: Actor, order: Order, slot: str) -> dict[str, str]:
if actor.role != "customer" or actor.user_id != order.owner_id:
raise PermissionError("not authorized for this order")
if slot not in {"morning", "afternoon", "evening"}:
raise ValueError("unsupported delivery slot")
return {
"status": "approval_required",
"order_id": order.order_id,
"requested_slot": slot,
}
actor = Actor(user_id="customer-17", role="customer")
order = Order(order_id="order-42", owner_id="customer-17", delivery_slot="afternoon")
print(propose_delivery_change(actor, order, "evening"))Returning a proposal is intentionally different from committing a change. A separate service can show the exact order and requested slot to the user, collect approval, and re-check current ownership and order state before writing. If approval expires or the state changes, ask the user again rather than assuming old consent still applies.
Make Approval Specific and Meaningful
Human approval should be reserved for actions whose impact justifies the extra step. Show the proposed operation, affected record, and important consequences in a trusted UI—not only in model-generated prose. Bind consent to the exact operation and arguments, record who approved and when, and invalidate it if the target or amount changes. Do not use a generic "continue" click as approval for a different action.
- Classify tools by impact, reversibility, and data sensitivity; default to no write capability.
- For approved actions, display the precise target and change using trusted application data.
- Bind approval to the specific user, operation, arguments, and expiry; prevent replay.
- Re-authorize and validate at commit time, then write an audit event with a correlation ID.
- Provide cancellation, rollback, or a support path appropriate to the action.
A human-in-the-loop control only helps when the reviewer has context and meaningful choice. If a workflow presents dozens of vague alerts or hides the action details, approvals can become routine clicks rather than informed checks. Test usability and bypass paths, not just the presence of an approval screen.
Contain Code or External Execution
Some agents analyze code, transform files, or run user-provided snippets. Do not execute untrusted code inside the main web application process. If execution is an essential product feature, use an isolated environment with a dedicated low-privilege identity, strict CPU and memory limits, timeouts, restricted filesystem access, and no network access unless the task specifically requires a controlled endpoint. Treat isolation as defense in depth, not proof that arbitrary code is safe.
For ordinary business tools, prefer allowlisted APIs and structured inputs over a general command runner. Keep secrets outside the agent context and sandbox, avoid mounting production credentials, and destroy or reset the environment after each job. Validate outputs before they re-enter trusted systems.
| Boundary | Control objective |
|---|---|
| Process / container | Separate untrusted work from application and host processes |
| Identity | Use a non-privileged, task-specific service identity |
| Resources | Bound runtime, memory, CPU, output, concurrency, and cost |
| Network | Deny by default; allow only reviewed destinations where required |
| Filesystem | Expose only task files; prevent access to host or other users’ data |
| Lifecycle | Reset state, remove temporary credentials, and preserve only approved outputs |
Bound Loops, Fail Closed and Keep Audit Evidence
An agent may repeat tool calls, call a downstream service slowly, or return malformed arguments. Bound total steps, tool calls, elapsed time, retries, output size, and model requests. Give every call a deadline and idempotency strategy appropriate to its operation. If authorization or policy services are unavailable, do not fall back to a broader tool or skip the check.
from dataclasses import dataclass
@dataclass
class ToolBudget:
max_calls: int
calls_used: int = 0
def consume(self) -> None:
if self.calls_used >= self.max_calls:
raise RuntimeError("tool-call budget exhausted")
self.calls_used += 1
budget = ToolBudget(max_calls=4)
for name in ["search_policy", "get_order_status"]:
budget.consume()
print("Allowed bounded call:", name)
print(f"Calls used: {budget.calls_used}/{budget.max_calls}")Record the actor, tool name, authorization result, approval reference, target identifier, and outcome in protected audit events. Avoid logging credentials, full sensitive arguments, or full prompts unless there is a justified, controlled need. Monitoring should help detect unexpected volume, repeated denials, and unusual sequences without turning logs into another source of data exposure.
Common Mistakes
Giving an agent a generic shell or database tool
Why: Broad capability creates a large blast radius and difficult-to-audit behavior.
Fix: Expose narrow business operations with strict schemas and server-side checks.
Trusting model-supplied identity or tenant values
Why: Those fields are untrusted arguments and can differ from the authenticated caller.
Fix: Inject principal context from the verified request and authorize every call.
Treating a confirmation prompt as the permission check
Why: The user may not see the exact effect, and authorization can change before execution.
Fix: Bind approval to the action and repeat authorization at commit time.
Assuming a container makes code execution harmless
Why: Isolation depends on configuration, host, network, secrets, and resource boundaries.
Fix: Use documented isolation controls, deny-by-default access, limits, and platform-specific review.
Reduce the Agency of a Delivery Assistant
Practice
Design least-privilege tools and an approval flow
- List the assistant’s requested capabilities and divide them into read, propose, and commit operations.
- Replace one broad tool with two narrow typed operations and specify server-derived identity and authorization checks.
- Design an approval record bound to a specific order and delivery-slot change, including expiry and replay protection.
- Write tests for wrong owner, malformed input, changed order state, authorization outage, and exhausted tool budget.
- Specify an isolated execution profile for an optional document-conversion task without mounting production secrets.
- Define audit fields that support investigation while minimizing personal data and prompt retention.
Hint: Use a fake order and customer. The model can propose a change, but only a verified user approval plus a fresh server authorization can commit it.
🎯 Key Takeaways
- ✓An agent’s tool call is an untrusted request; the tool service must validate arguments and authorize the actor independently.
- ✓OWASP LLM06 Excessive Agency concerns excessive functionality, permissions, or autonomy.
- ✓Prefer narrow, least-privilege tools and separate read, propose, and commit operations.
- ✓Human approval should display and bind to the exact operation, target, and arguments, followed by authorization at commit time.
- ✓Sandbox untrusted execution with documented isolation, resource limits, restricted network and filesystem, and no production secrets.
- ✓Bound agent loops and record privacy-aware audit evidence; fail closed when critical policy checks are unavailable.
- Reference: OWASP Top 10 for LLM Applications (2025), especially LLM06 Excessive Agency and LLM05 Improper Output Handling.
- Reference: MITRE ATLAS current knowledge base for AI adversary techniques and mitigations; adapt relevant scenarios to your agent’s actual tools.
Next
What comes next
You have constrained the agent’s actions. Next, protect sensitive information in prompts, logs, and generated outputs.
Lesson 06 · Sensitive Data, Privacy and Output Handling →HELP US IMPROVE
How was this lesson?
Your note is shared with the Chaduvuko team for review. Please don’t include personal or sensitive information.
Spotted a mistake or something unclear? Suggest a fix on GitHub or email us. Select the text first and it is quoted for you.
Discussion
0Have a better approach? Found something outdated? Share it — your knowledge helps everyone learning here.
