Import AI 420: Prisoner Dilemma AI; FrontierMath Tier 4; and how to regulate AI companies

Welcome to Import AI, a newsletter about AI research. Import AI runs on lattes, ramen, and feedback from readers. If you’d like to support this, please subscribe.

AI pentesting systems out-compete humans:
…Automated pentesting…
AI security startup XBOW recently obtained the top rank on HackerOne with an autonomous penetration tester - a world first. "XBOW is a fully autonomous AI-driven penetration tester," the company writes. "It requires no human input, operates much like a human pentester, but can scale rapidly, completing comprehensive penetration tests in just a few hours."

What they did: As part of its R&D process, XBOW deployed its automated pen tester onto the HackerOne platform, which is a kind of bug bounty for hire system. "Competing alongside thousands of human researchers, XBOW climbed to the top position in the US ranking," the company writes. "XBOW identified a full spectrum of vulnerabilities including: Remote Code Execution, SQL Injection, XML External Entities (XXE), Path Traversal, Server-Side Request Forgery (SSRF), Cross-Site Scripting, Information Disclosures, Cache Poisoning, Secret exposure, and more."

Why this matters - automated security for the cat and mouse world: Over the coming years the offense-defense balance in cybersecurity might change due to the arrival of highly capable AI hacking agents as well as AI defending agents. This early XBOW result is a sign that we can already develop helpful pentesting systems which are competitive with economically incentivized humans.
Read more: The road to Top 1: How XBOW did it (Xbow, blog).

***

AI personalities revealed by Prisoner Dilemma situations:
…Gemini is 'strategically ruthless', while Claude is 'the most forgiving reciprocrator'...
Researchers with King's College London and the University of Oxford have studied how AI systems perform when playing against one another in variations of iterated prisoners' dilemma games, the classic game theory scenarios used to assess how people (and now machines) reason about strategy. For this study they look at models from Google, OpenAI, and Anthropic, and find that "LLMs are highly competitive, consistently surviving and sometimes even proliferating in these complex ecosystems".

What they did: The paper sees the researchers study Google and OpenAi models in a few variations of prisoner dilemma games, and then they also conduct a tournament where AI systems from Google, OpenAI, and Anthropic are pitted against a Bayesian algorithm. "In all we conduct seven round-robin tournaments, producing almost 32,000 individual decisions and rationales from the language models," The study ...