Final-year project
In progressSmall Agent Safety Evaluation
When a small AI agent fails a harmful task, it might have refused, or it might just not be capable of doing it. Most safety scores can't tell those apart. I'm building an open-source tool that pairs each harmful task with a harmless one using the same tools, so you can see which.
I'll use it to compare three sizes of Qwen3.5 (4B, 9B and 27B), with reasoning on and off. It's built on Inspect AI and AgentHarm, test-first, with a CLI, a local dashboard and PDF reports.
- Python
- Inspect AI
- AgentHarm
- Ollama
- pytest