I Found a CVE With a Scanner That Misses Almost Everything

The number I publish first

In July I benchmarked the scanner against five third-party MCP servers: notion, neon, qdrant, firecrawl, and airtable. Ground truth was frozen to disk by five independent audits before the first scan ran, so I could not tune toward the answer.

It found none of it. Zero true positives out of 58 findings. Pooled recall 0 of 69. Even restricted to the three vulnerability classes the scanner actually implements, 0 of 7. The best case was mcp-server-qdrant, where every Python file loaded with full ASTs and every detector was eligible, and it found 0 of 9, including a path traversal a blind auditor reproduced live. On firecrawl it reported twelve hardcoded-secret findings, all of them synthetic test fixtures, while missing a real live-shaped Airtable token.

I pulled the paid one-shot audit tier that same day and blocked every outreach draft from citing a finding this scanner produced. Neither has been reinstated.

It also found a real one

Separately, as a dogfooding exercise, I had pointed it at popular public MCP servers. One detector flagged a code generation path in awslabs/dynamodb-mcp-server: a caller-influenced identifier rendered into generated source with Jinja autoescape disabled. Generated code is an execution surface, and an identifier position is not a place you can rely on escaping to save you.

I reported it to AWS on July 23rd. What happened next, with dates:

• 2026-07-23 reported by email to the AWS security team.
• 2026-07-24 acknowledged, under investigation.
• 2026-08-13 a Vulnerability Disclosure Program contact assigned.
• 2026-09-03 AWS confirmed the fix shipped in awslabs.dynamodb-mcp-server 2.1.6.
• 2026-09-08 the advisory published.

It is GHSA-35jj-hwvm-792x, CVE-2026-85654, "Code injection in the CDK generator in Amazon awslabs.dynamodb-mcp-server". Rated High, CVSS 7.8. Affects >= 2.0.10, <= 2.1.5, patched in 2.1.6. I am credited on it as coordinator.

One note on that link. The advisory is repository-scoped and, at the time of writing, is not mirrored into the global GitHub Advisory Database, so the github.com/advisories/ form of the URL returns a 404. The link above is the one that resolves.

The part I want to be precise about

Precision is the entire point of the piece, so here are the three things this is not.

The upstream fix was more complete than my finding. I flagged two interpolation sites. The patch that shipped covers nine, and it correctly separates identifier positions from string-literal positions, which my report did not. I did not write that patch and I do not claim it.

Whether the patch author worked from my report is not something I can see. AWS stated which release carried the fix for the reported issue. The internal path from report to pull request is theirs, not mine, and I claim it in neither direction.

One CVE does not move a 0-of-69 recall number. It is one finding, not a hit rate. Pooled recall has not been re-scored against that frozen ground truth since July, so there is no current recall figure and I do not quote one. If you ever see me cite this CVE without the recall number next to it, I am selling you something.

So what is it evidence of

Two things, and neither of them is "the scanner is good."

The first is that a narrow detector, aimed at one specific injection shape, can find something real in production code maintained by people far better resourced than me. Breadth and depth are different products. A tool that checks one thing properly is not competing with a tool that checks everything badly, and the industry keeps pricing them as though it is.

The second is the one I actually care about. The useful discipline was never building the tool. It was measuring it honestly enough to know which of its findings deserved a stranger's attention, and then being willing to publish the number that makes the tool look bad. The 0 of 69 is why I only sent one report instead of a list. A tool I had not measured would have produced a confident-sounding email full of synthetic test fixtures, and the person on the other end would have been right to ignore it.

That is the same discipline I bring to a client server, and it is a good deal less comfortable than a marketing page. The scanner is public, with its test suite and its recall result stated in its own README rather than in a footnote.

If you are shipping an MCP server

• Treat generated code as an execution surface. Template autoescape is not a control when the injected value lands in an identifier position rather than a string literal.
• Ask any scanner you rely on for its recall number against a ground truth it did not choose. If it does not have one, it has not been measured, and an unmeasured detector's silence is silence, not a clean bill.
• Count the findings you can reproduce, not the findings you can produce. A larger denominator is not a capability.