Testing Strix, part 1: a pentester agent, not a linter

I’ve used plenty of static security scanners over the years. They’re fine for quick checks, but mostly they just tell you what looks suspicious.

The real headache is what comes next. Is the flagged code actually reachable? Can someone exploit it? Does it cross a trust boundary that actually matters in your architecture?

That’s where Strix caught my attention.

It’s an open-source LLM agent designed to behave like a pentester rather than a static linter. Instead of just flagging code patterns, it runs recon, maps out the attack surface, tests hypotheses, and attempts real exploitation to prove its findings.

The key difference comes down to pattern matching versus actual proof.

A standard scanner points at a line and says, “This looks like an injection”. An agent like Strix sends a payload, inspects the response, and hands you a reproducible proof-of-concept showing how and why it worked.

I tested it against a web app I’ve been building, running two separate scans: one against the live application and another directly against the source code repository. The results were surprisingly different between the two modes.

The output comes back as clean markdown for each finding (complete with severity, impact analysis, and a PoC), plus JSON and SARIF files if you want to pipe results into your CI/CD pipeline.

What makes this approach interesting isn’t just “AI in security”. The main bottleneck in application security has always been triage, not generating potential leads. A tool that provides attempted proofs alongside findings changes how we review results, even when those proofs turn out to be wrong.

And as I found out, they often are. More on that in part 2 and part 3.

Strix on GitHub: github.com/usestrix/strix