Set a narrow job
Choose a mailbox and define categories such as billing, technical support, sales, complaint, and urgent safety issue. Give each category an owner, service target, and escalation rule. Decide whether the agent may add labels, create tickets, or only recommend actions. Start with read access and draft creation; do not grant autonomous sending or deletion for a first release. Identify sensitive messages that should bypass model processing or receive stricter handling.
Write routing examples
Collect consented or sanitized examples, including messages with two intents, forwarded threads, attachments, different languages, and vague subject lines. Have support staff label category, urgency, responsible team, and whether a reply is needed. Resolve disagreement before using examples as ground truth. Split by thread or customer so a nearly identical continuation does not appear in training and test. A category list without real boundary examples will produce inconsistent routing.
Treat mail as untrusted input
An email can contain instructions that pretend to be from an administrator, request data from another account, or ask the agent to forward secrets. Treat its body and attachments as task data, not operating instructions. Enforce tool permissions outside the model and keep tenant boundaries in the mail API. OWASP recommends testing prompt injection and unauthorized tool use in agents. Include such messages in the evaluation set and verify no prohibited side effect occurs.
Review the drafted action
Show the reviewer the original message, proposed category, urgency, destination, draft reply, and cited facts. Make correction fast and record why a route changed. A person should approve before a reply is sent or a consequential ticket is closed. Track routing precision and recall by category, review time, missed urgent cases, and correction rate. A fluent draft that assigns the wrong owner is still a triage failure; evaluate the complete workflow.
Expand only after evidence
Run a shadow pilot beside the existing mailbox process, then permit a small set of reversible labels or ticket drafts. Compare against the baseline and review every urgent miss. Version the category rules, examples, model, and tool permissions together. Add a new category only with labeled boundary cases and a responsible team. Keep an easy disable switch and a manual queue for outages. The measured benefit should be faster correct routing, not simply more automated clicks. Keep the tested task, tool permissions, and expected side effects in one versioned release record. Re-run both ordinary and hostile cases when a model, prompt, tool, or approval rule changes. Connecting a triage agent to a real mailbox, ticketing system and review screen is the kind of integration we describe on our AI agent development page.
Technical references: OWASP AI agent security, Anthropic agent evaluations. Related guide. Check the cited documentation against your own data, permissions and release version before applying this design.