OpenAI, in collaboration with crypto investment firm Paradigm, launched EVMbench on Monday, a new benchmark designed to evaluate the capabilities of artificial intelligence in securing smart contracts on blockchain networks. The tool aims to measure AI’s ability to detect, exploit, and patch vulnerabilities within these contracts, which underpin a growing range of decentralized financial applications.
Smart contracts, self-executing agreements written into code, are a foundational element of platforms like Ethereum. Their immutable nature – meaning they cannot be altered once deployed – makes security paramount. Flaws in smart contract code can lead to significant financial losses, as vulnerabilities can be exploited by malicious actors.
EVMbench utilizes 120 vulnerabilities sourced from 40 audits and security competitions, according to Bankless, a crypto news and research platform. The benchmark assigns a percentage-based score to AI agents based on their performance in auditing contracts, patching vulnerabilities while maintaining functionality, and exploiting weaknesses in vulnerable contracts.
“Smart contracts routinely secure $100B+ in open-source crypto assets,” OpenAI stated in a blog post. “As AI agents improve at reading, writing, and executing code, it becomes increasingly vital to measure their capabilities in economically meaningful environments, and to encourage the use of AI systems defensively to audit and strengthen deployed contracts.”
The launch of EVMbench comes as the use of AI in cybersecurity gains momentum. OpenAI and Paradigm hope the benchmark will establish a standardized method for assessing AI’s effectiveness in identifying and mitigating risks within the complex world of blockchain technology. While the developers acknowledge the benchmark doesn’t fully capture the challenges of real-world smart contract security, it provides a measurable baseline for evaluating emerging AI agents designed for the crypto space.
CoinDesk reported that the tool is OpenAI’s attempt to determine whether modern AI systems are up to the task of preventing smart contract issues. The initiative represents a deepening of OpenAI’s involvement in crypto security, seeking to leverage AI to enhance the integrity of blockchain applications.
