Excessive Agency in LLM Applications
detect, understand, remediate

Tool grants and function calls

A registered function the model can invoke. send_email, refund_order, create_user, update_record, run_query, schedule_meeting, deploy_app. Every entry in the tool registry is a delegated authority. The set of tools the agent can reach is the upper bound on the blast radius of any injection that lands.

Database and storage handles

The model can read or write a row through a tool. The handle runs under a service identity that often has broader permissions than the requesting user. A read tool exposed without per-user scoping returns data across tenants on the first injection that asks the right question.

Outbound HTTP and webhook actions

An agent fetches a URL, posts to a webhook, or calls a partner API. The fetcher inherits whatever credential the tool was registered with. Where the credential is broad (workspace-wide token, admin API key), one injection turns into a cross-system action.

Code execution and notebook runners

A data-analysis assistant, a research agent, or a CI helper hands the model a code-execution tool. The sandbox runs with environment variables, mounted volumes, and a network namespace. Each one is an authority the model now wields without an authenticated user behind it.

File system and document writes

A report-generation tool writes a file to a shared drive. A document agent updates a wiki page. A code agent commits to a repository. The tool grant is the write authority on the surface; the actor identity captured in the audit log is often the model's, not the user's, which breaks accountability.

Privileged or destructive operations

Tools that delete data, revoke access, charge cards, ship orders, change configuration, rotate keys, or deploy production. Each one is a separate authority that should be separately granted, separately scoped, separately logged, and (for most of them) separately confirmed by a human before the model can fire it.

Agent-to-agent delegation

A parent agent calls a child agent that has its own tool set. The child inherits or is granted authorities the parent did not have. The composition turns excessive agency into an emergent property the engineering team did not register in any single tool list.

Persistent memory writes

A long-running agent stores facts, preferences, or instructions into long-term memory. A write authority on memory is an authority to plant instructions for every future session that retrieves the same memory. The discipline that internal teams apply to persistent state on the rest of the application applies here too.

Registering the full SDK instead of the subset the feature needs

Engineering teams import a provider SDK that exposes every operation, then register the whole surface as agent tools. The summariser ends up with send_email because the SDK ships send_email. Tool selection at registration time is the cheapest place to apply least privilege.

Service-account identity reused across tenants

A single service identity holds the credentials every tool call uses. Per-user scoping was deferred. The injection turns the user into the service identity. Per-user OAuth, per-user scoped tokens, or per-request impersonation is the operational fix.

Approval gates skipped to keep the product fast

Human approval was framed as friction. The team removed it. The model now writes, deletes, charges, deploys, and ships on its own judgement. The first injection that lands becomes a production action because no humans were in the loop to stop it.

Trusting the model to validate its own outputs

The application asks the model whether the call is appropriate before firing it. The check is the same untrusted source. Schema validation, allow-listing, deny-listing, and authorisation belong in the application code, not in another model call.

Long-lived broad credentials in tool registrations

A bearer token with workspace admin scope, a cloud root key, or a long-lived OAuth refresh token sits inside the tool config. The credential should be the narrowest scope the feature can do its job under, with the shortest lifetime the workflow can tolerate.

No structured log of what the agent actually did

The team relies on the LLM provider's trace as the record of truth. The trace is opaque, retention-bounded, and not under the team's control. The audit record needs to live on the application's operating surface alongside every other security event.

Automated detection

• SecPortal's code scanning runs against connected repositories and flags agent tool registrations where the credential is long-lived, the scope is workspace-wide or admin-wide, the parameter schema is loose or absent, the call lacks an explicit per-call authorisation read against the user identity, or a high-impact tool (write, delete, send, charge, deploy) fires without a documented human-approval gate
• Authenticated scanning drives the LLM-backed endpoint with prompts that ask the model to invoke each registered tool with out-of-policy arguments, observes whether the call fires, captures the request, the parameters, and the resulting downstream state change as evidence on the finding
• External scanning discovers exposed agent webhooks, tool-callback URLs, and public agent endpoints reachable from the public internet that may invoke tools without authentication or under a shared identity
• Continuous monitoring re-runs the tool-abuse probe on a defined cadence so a new tool registration, a permission scope widened in a deployment, or a removed approval gate is caught against the previous baseline rather than waiting for the next pentest cycle

Manual testing

• Enumerate every registered tool the agent can invoke, the parameter set, the credential the tool runs under, the identity the call is attributed to, and whether each call is logged on the application's operating record
• For each tool, craft a prompt that asks the model to call the tool with arguments outside the registered allow-list (other tenants, other users, larger amounts, broader scopes, different recipients) and record which calls fire and what side effect the call produces
• For agent chains, map the call graph from parent agent to child agent to tool, and confirm whether composition through delegation creates an emergent authority no single registration captured
• For high-impact tools (write, delete, send, charge, deploy), confirm the presence of an explicit human-approval gate and that the gate cannot be skipped through prompt content alone
• Read the audit log for the test session and confirm that each tool invocation captures the prompt that triggered it, the parameters the model produced, the actor identity attribution, the outcome of the call, and the time stamps a responder would need for post-incident reconstruction

Finding record with prompt, tool, parameters, and outcome

The finding captures the original prompt, the tool the agent invoked, the parameter set the model produced, the actor identity the call ran under, and the observed state change (the email sent, the refund processed, the record updated, the file written, the API called). The evidence is what AppSec, product security, and AI engineering need to reproduce the abuse against the same surrounding code path.

Code scanning across tool registrations

Code scanning runs against connected GitHub, GitLab, and Bitbucket repositories. Findings surface at tool-registration sites where the credential scope is too broad, the parameter schema is loose or absent, the authorisation check reads the model rather than the user, or a high-impact tool lacks a documented human-approval gate. The remediation lands at the registration site rather than at a perimeter filter.

Authenticated scanning with tool-abuse payloads

Authenticated scanning runs against the LLM-backed endpoint with a curated set of tool-abuse prompts under a real session. Each probe records whether the tool call fires, the parameters the model produced, and the downstream state change, so the finding ties to a concrete authority misuse rather than to a model output alone.

Continuous monitoring against tool registration drift

Continuous monitoring re-runs the tool-abuse probe on the configured cadence. A new tool registration, a permission scope widened in a deployment, or a removed approval gate that re-opens a previously closed finding shows up against the baseline rather than waiting for the next pentest cycle.

Retest after the remediation ships

Once the fix deploys (the credential is narrowed, the schema is tightened, the approval gate is added, the per-user authorisation is wired), a targeted retest replays the original tool-abuse prompt against the new registration and records the post-fix response on the finding. The finding closes against the evidence rather than against a developer assertion that the registration is now safe.

AI-assisted writeups with explicit honest scope

AI reports generate the writeup, the executive summary, and the developer-facing reproduction steps from the finding record. The narrative stays within the verified evidence on the finding (the prompt, the tool call, the parameters, the outcome) and does not invent guardrails, sandbox behaviour, or runtime tooling the product does not have.

Activity log for the tool-invocation audit trail

The workspace activity log captures the security-side audit trail: who created the finding, who suppressed it, who closed it, who approved a related exception, and when the retest fired. Pair the workspace activity log with the engineering-side per-tool invocation log the application writes against every agent call so post-incident reconstruction reads from both records.

Finding overrides for documented exceptions

Where a tool grant is a deliberate test fixture or a sanctioned internal-only surface, finding overrides record the suppression rationale, the owner, and the expiry on the finding itself. The exception lives on the operating record rather than in a parallel spreadsheet.

Compliance tracking pairs the fix to control evidence

Compliance tracking maps excessive-agency findings to the controls that read against them (ISO 27001 A.5.15 access control, A.8.2 privileged access rights, A.8.3 information access restriction, A.8.18 use of privileged utility programs; SOC 2 CC6.1 logical access, CC6.3 access provisioning; NIST SSDF PW.5 secure coding; NIST AI RMF Map and Manage; ISO/IEC 42001 AI management system controls). The same finding feeds the engineering ticket and the auditor evidence pack.