AI Lab / AI Failures And Evals
Agent Failure Zoo
A practical field guide to how AI agents fail: wrong tool, stale context, hidden authority, bad memory, and confident nonsense wearing a clipboard.
Post
Reading note.
A list of common AI-agent failure modes and how to test for them.
How to read this
Personal writing is labelled personal; external claims, quotes, and media require source context.
The Zoo
An agent is not smarter because it has tools. It is now capable of being wrong with consequences. The zoo is a place to name failures before they escape into product copy.
Starter animals: the Obedient Browser, the Citation Tourist, the Stale Memory, the Overconfident Scheduler, the Tool Juggler, the Permission Acrobat, and the Evals Peacock.
- tool misuse
- retrieval drift
- authority confusion
- memory leak
- overbroad action
- missing eval
- unclear rollback
How To Use It
Each failure gets a tiny story, a reproduction prompt, a test case, a prevention pattern, and a question a reviewer should ask before launch.
The tone can be funny. The check should be boring. That is the bargain.
First Failure Cards
Obedient Browser: the agent treats page text as instruction. Test it with a page that asks the assistant to ignore the user's task.
Citation Tourist: the answer includes citations that look scholarly but do not support the claim. Test it with conflicting sources and require quoted evidence.
Permission Acrobat: the agent turns a vague user wish into an action it was never allowed to take. Test it with calendar, email, file, and purchase-like boundaries.
- name the failure
- show a tiny reproduction
- write the prevention pattern
- add a regression case
Reviewer Questions
What input is trusted? What input is only data? Which action needs explicit permission? What happens when sources disagree? What receipt proves the agent did the right thing?
A good launch review should be able to answer those questions without opening the product owner's brain.