A2A Check
DEStart
All posts

10 November 2026

57% correct, 11.8% secure: why working agent code is not secure code

The tests are green and the feature works. A recent study shows why, for code written by AI agents, that says little about security.

The number

Researchers led by Songwen Zhao examined how secure the code is that AI agents write for real tasks. Their benchmark, SusVibes, holds 186 tasks from real open-source projects. For each task, human developers had once committed a vulnerable implementation.

They tested 12 widely used agent setups with frontier models. The authors' finding: all of them perform poorly on security. With SWE-Agent and Claude 4 Sonnet, 57% of the solutions were functionally correct, but only 11.8% were secure.

Put differently: out of 100 solutions, 57 worked and barely 12 were secure.

Hints in the prompt do not help

The obvious idea is to warn the agent: "Watch out for vulnerabilities." The study tested exactly that. Adding hints about possible vulnerabilities to the task did not fix the problem.

This matches other work on agents, such as MCP-SafetyBench (ICLR 2026): a friendly sentence in the prompt is not control. Control comes from outside the model, from checks that run regardless of what the agent "wants".

What this means for your team

When an agent writes code, "the tests pass" is only half the evidence. Three things need to be added:

  1. A fixed end with a check. The agent does not report "done" before tests, build and linter have run. This is written in the instruction file and ideally enforced technically, for example by a hook or the CI.
  2. A security check that does not depend on the agent. Static security analysis and a dependency check in CI find flaws the agent does not see.
  3. Human review where it is risky. Authentication, permissions, external input, database queries. A second look always pays off here.

An example

An instruction file that only asks for function:

Implement the issue and make the tests green.

One that builds in control:

Goal: implement the issue, one pull request.
Success: tests, build and linter pass; the security check in CI reports nothing new.
Changes to authentication, permissions or database queries: flag them in the pull request, I review them myself.
Never alone: merge a pull request.

Check it yourself in three steps

  1. Read your agent's instructions: do they say how it recognises success, or only what to build?
  2. Look at your CI: does it run a security check, or only tests?
  3. Take your agent's last pull request that handles external input and read it with one question: what happens with malicious input?

What A2A Check looks at

A2A Check does not check the code your agent writes. It checks the configuration that decides how your agent works. Among other things, it reports when a job does not say how it can tell that it succeeded, when there are no test cases for the agent's behaviour, or when nothing says what the agent must never do alone.

The study shows that the model alone cannot be relied on. Check whether your configuration builds in control: a2a-check.com

Sources