BenchJack: Scans AI Agent Benchmarks for Hackability Vulnerabilities
BenchJack audits AI agent benchmarks to detect hackability flaws like leaked keys, unsafe evaluations, and prompt injections that let models cheat without real capability. Designed for developers and researchers, it employs static tools including Semgrep and Bandit plus AI-driven analysis with Claud