Google’s Project Zero has introduced an initiative called Naptime, a new framework that will be used to enable a large language model to conduct vulnerability research.
Initially started in mid-2023 and aims to improve vulnerability discovery approaches, with a particular focus on automating variant analysis and the objective is to enable an LLM to perform vulnerability research that closely mimics human security experts’ iterative, hypothesis-driven approach.
The framework’s architecture is centered around the interaction between an AI agent and its set of specialized tools designed to mimic the workflow of a human security researcher and a target codebase.
These tools include:
- The Code Browser enables the agent to navigate through the target codebase.
- The Python enables the agent to run Python scripts in a sandboxed environment
- The Debugger allows the agent to interact with the program and observe its behavior under different inputs.
- The Reporter provides a structured mechanism for the agent to communicate its progress
- The Controller verifies and also allows the agent to abort the task when unable to make further progress, preventing stagnation
Researchers stated that the framework is model-agnostic and backend-agnostic and builds on a set of guiding principles established by Google’s Project Zero to improve the performance of multi-purpose LLMs in vulnerability discovery.
These principles were developed following the launch by security researchers at Meta of CyberSecEval2, the latest LLM benchmark for discovering and exploiting memory safety issues.
Project Zero researchers carried out two series of the CyberSecEval2 tests, ‘Advanced Memory Corruption’ and ‘Buffer Overflow,’ with GPT 4 Turbo as the AI agent and the rest of the Naptime tools. They achieved new top scores of 1.00 on the ‘Buffer Overflow’ tests and 0.76 on the ‘Advanced Memory Corruption’ tests
With the right tools, current LLMs can really start to perform vulnerability research! However, there’s a large difference between solving isolated capture the flag-style challenges without ambiguity and performing autonomous offensive security research,
They believe the security community will also need to develop more difficult and realistic benchmarks to efficiently monitor the progress of such initiatives.
