AI Agent Security

Question 1

Can prompt injection make an AI agent send money?

Accepted Answer

Yes, if the agent can pay and treats text it reads as instructions. A malicious web page, email or document can carry instructions that steer the agent, which NIST calls agent hijacking. The defense is to make payments depend on checks outside the model, such as limits and approvals, rather than on the model resisting the text.

Question 2

How do you protect an AI agent's wallet from prompt injection?

Accepted Answer

Keep the wallet's authority outside the model's reach. Enforce spending limits, allowed destinations and approval thresholds in the wallet or a policy service the agent cannot edit, give the agent a narrow session key instead of the main key, require human approval for large or unusual payments, and treat external content as untrusted.

Question 3

What are the biggest security risks of AI agents that handle payments?

Accepted Answer

Prompt injection and hijacking that steer the agent into paying, stolen or leaked keys and API credentials, malicious tools or MCP servers, manipulation by the sites it visits, overly broad permissions and payments that cannot be reversed. Most come down to the agent holding more authority than one compromised step should allow.

Question 4

How do you stop an AI agent from leaking API keys or private keys?

Accepted Answer

Never put keys where the model can see them: not in prompts, context or tool outputs. Keep them in a secrets manager or a secure signing service, and let the agent request signatures or use short-lived tokens instead. Rotate secrets, scope each one to the least access needed and audit every use.

Question 5

What happens if an AI agent's credentials are stolen?

Accepted Answer

The attacker can act as the agent until the credentials are revoked: spend within whatever limits the credentials allow, call its APIs and impersonate it to other services. That is why limits must sit outside the credentials, secrets should be short-lived, and you need a fast way to revoke and rotate them.

Question 6

What is a kill switch for AI agents and how should it work?

Accepted Answer

A kill switch is a way to stop an agent from acting at once: revoke its keys, sessions and payment permissions, switch its policy to deny every request and halt its running tasks. It must work without the agent's cooperation, be usable by an authorized person at any time and leave a log of what was stopped.

Question 7

How do you secure tool calls in an MCP server that touches money?

Accepted Answer

Obtain explicit user consent before tools run, as MCP's security principles call for, validate every tool input, authorize each call against the user's limits on the server side, accept only tokens issued for your server and request the narrowest scopes. Pin tool versions and log every call that moves money.

Question 8

Can a malicious website trick a shopping agent into a bad purchase?

Accepted Answer

Yes. Hidden or misleading content on a page can carry instructions the agent follows, an attack known as indirect prompt injection, and fake listings or prices can mislead it. Limit what the agent can buy without approval, verify merchants and prices against trusted data, and keep payment checks outside the model.

Question 9

Should an LLM ever hold signing authority directly?

Accepted Answer

Not directly. The model should be able to request a signature, not perform one with an unlimited key. Put a policy check between the request and the signer, use keys with narrow permissions such as session keys, and keep the full-authority key away from the agent. A model can be manipulated; the signer's rules should not be.

Question 10

What is a blast radius limit for an AI agent?

Accepted Answer

A blast radius limit caps how much damage one compromised agent or credential can do: a maximum spend, a limited set of destinations, short-lived credentials and no access to other agents' funds or data. It assumes a compromise will eventually happen and makes sure its cost stays bounded.

Question 11

How do you sandbox an AI agent that can move funds?

Accepted Answer

Run it with only the access it needs: a separate wallet or session key holding a small balance, network access limited to approved services, no access to other secrets, and every payment passed through a policy check. Test it on a testnet first. MCP's guidance recommends running servers sandboxed, with minimal default privileges.

Question 12

How do you detect an AI agent behaving abnormally?

Accepted Answer

Build a baseline of what the agent normally does, including its usual payees, amounts, timing and tools, and alert on deviations: new destinations, sudden spikes in volume, payments just under limits or repeated denials. Review alerts quickly, and let high-risk anomalies pause the agent automatically.

Question 13

How do you prevent replay attacks on agent payment requests?

Accepted Answer

Make every payment request unique and short-lived. Include a nonce plus creation and expiry times in the signed request, reject any request already seen or outside its time window, and use idempotency keys so a retried payment executes only once. HTTP Message Signatures support created, expires and nonce parameters for this.

Question 14

What is the principle of least privilege for AI agents?

Accepted Answer

It means giving an agent only the minimum access and authority it needs for its task, and nothing more. NIST defines least privilege as granting each entity the minimum resources and authorizations it needs to perform its function. For payment agents, that means narrow scopes, small budgets and limited destinations.

Question 15

How do you red-team an AI agent that makes payments?

Accepted Answer

Attack it the way adversaries would before launch: plant malicious instructions in pages, emails and tool outputs, try to make it exceed limits, pay new destinations or leak keys, and repeat attacks many times. NIST's work on agent hijacking stresses adaptive attacks and multiple attempts. Fix whatever gets through, then test again.

Question 16

What is agent hijacking?

Accepted Answer

Agent hijacking is when an attacker takes control of what an AI agent does by inserting malicious instructions into data it reads, such as an email, file or website, steering it into harmful actions it was never meant to take. NIST describes it as a form of indirect prompt injection.

Question 17

What is tool poisoning in MCP servers?

Accepted Answer

Tool poisoning is an attack in which malicious instructions are hidden in an MCP tool's description, invisible to the user but read by the AI model, which may then follow them, for example to leak data. Mitigations include showing full tool descriptions, pinning tool versions and checking them against a hash.

Question 18

How do you secure an AI agent's API keys in production?

Accepted Answer

Store them in a secrets manager, never in code, prompts or logs. Give each agent its own key with the least access it needs, prefer short-lived or dynamic credentials, rotate keys regularly and audit every use. Revoke a key immediately when an agent is retired or behaves unexpectedly.

Question 19

What is indirect prompt injection and why does it matter for shopping agents?

Accepted Answer

Indirect prompt injection is when instructions reach a model through content it retrieves, such as a web page, review or document, instead of from the user. Shopping agents read untrusted pages constantly, so a product page can try to make the agent ignore the user's rules or buy something else.

Question 20

How do you stop a merchant site from manipulating an AI agent?

Accepted Answer

Treat merchant pages as untrusted data, not instructions. Keep what the user asked separate from what the page says, verify price and product against structured data or trusted feeds, require confirmation when terms change, and enforce payment limits outside the model, so even a manipulated agent cannot overspend.