AI Lab / AI Failures And Evals
Break The LLM Challenge
A playful challenge shelf for prompt injection, hallucination, tool misuse, tokenization, eval gaps, and other AI failure modes.
Resource
Use case, then evidence.
Small challenges that teach how AI systems fail.
- Freshness
- seed / reviewed 2026-08-12
- Sources
- 2 attached now; source plan has 3 checks.
- Next useful action
- Create challenge taxonomy
Why It Is Fun
People learn AI safety faster when the failure is visible. A small game can teach trust boundaries, retrieval risk, prompt injection, brittle evals, and exact-computation gaps without sounding like a policy memo.
The vibe should be mischievous but responsible: break the toy, not real systems.
Challenge Types
Planned lanes include museum-guard prompt injection, citation trap, tokenizer trap, malicious spreadsheet, hidden instruction in a PDF, overconfident math answer, and tool-use permission puzzle.
First Three Levels
Level one is a tokenizer trap: ask for an exact character-level answer and make the learner inspect why the model may stumble.
Level two is a hostile web page: the learner must separate user intent from untrusted retrieved text.
Level three is a tool-permission puzzle: the model has enough information to act, but not enough authority.
- exactness
- trust boundary
- source citation
- tool permission
- regression case
Scoring Rule
The score should reward the explanation, not only the exploit. A good answer names the failure, names the boundary, proposes a test, and explains what the product should block.
That keeps the challenge playful without training people to confuse real security work with poking random systems.