Depthfirst Achieves State-of-the-Art Vulnerability Detection with RL-Trained Agent dfs-mini1
Key Takeaways
- ▸dfs-mini1 achieves state-of-the-art performance on EVMBench Detect at pass@8, demonstrating the effectiveness of RL post-training for security vulnerability detection
- ▸Restricted context windows and enforced constraints can improve reasoning quality, as the agent learns to focus on task-relevant information rather than relying on irrelevant tool outputs
- ▸Custom RL training infrastructure with diverse, domain-specific environments (50% larger codebase scope than evaluation sets) enables better generalization across multiple smart contract languages
Summary
Depthfirst has unveiled dfs-mini1, a specialized security agent trained through reinforcement learning to detect vulnerabilities in smart contracts with state-of-the-art performance. The agent achieved Pareto optimality on OpenAI's EVMBench Detect benchmark, which evaluates vulnerability detection recall on high-severity smart contract flaws that could result in irreversible financial loss. The company built custom infrastructure on Kubernetes to run thousands of sandbox environments for training, using historical smart contract audits from multiple platforms spanning Solidity, Rust, Cairo, and Vyper.
Depthfirst's approach highlights how strategic constraints can improve AI agent performance. The team restricted dfs-mini1 to a 32k context window—well below the base model's native capacity—and implemented summarization-based context compaction strategies to handle large codebases efficiently. Through training, the agent learned to use its turns more effectively and compress information more efficiently. The company also discovered that exposing only low-level primitives (shell commands) rather than higher-level tools prevented the agent from over-relying on static analysis tools that generated false positives.
- Exposing low-level primitives rather than specialized security tools allows agents to develop flexible detection strategies without anchoring to fixed methodologies
Editorial Opinion
Depthfirst's approach demonstrates an important principle in AI development: well-designed constraints and domain-specific training can substantially improve agent performance beyond what generic scaling provides. The deliberate choice to use low-level primitives and restrict context windows—decisions that might seem counterintuitive—actually enhanced the agent's reasoning and prevented failure modes. This work suggests that security-critical AI applications benefit from careful architectural choices tailored to the problem domain rather than simply increasing model size or capability.
