Agent-on-agent attacks happen when one automated agent manipulates another agent’s instructions, tools, data, or trust assumptions. The attacker might be a malicious document, a compromised tool response, a poisoned memory entry, or an external agent that sends carefully crafted content.
The most important design principle is to separate untrusted text from authority. An agent should not treat retrieved content as a command, and one agent’s output should not automatically grant another agent permission to send email, modify files, or spend money.
Use least-privilege tools, explicit approval gates, schema validation, provenance tracking, and short-lived credentials. Give each agent a narrow role. For example, a research agent may collect sources but should not publish directly. A writing agent may draft content but should not alter billing settings.
Log every tool call with the requesting agent, input, output, user approval, and resulting change. Add adversarial tests for prompt injection, data exfiltration, conflicting instructions, replayed messages, and unsafe delegation.
A secure architecture assumes that any agent can be wrong or compromised. Human approval is not needed for every harmless step, but irreversible or external actions should have a clear boundary. Reliability comes from controlled permissions and observable workflows – not from assuming that a model will always recognize a malicious instruction.