What Is AI SAST? What the LLM Actually Does

TL;DR: AI SAST is static application security testing in which a large language model (LLM) helps find, triage, fix or confirm vulnerabilities in source code. In AI-assisted tools, a rule-based engine produces the findings; in AI-native tools, the model finds them. The main gain is coverage: a model can catch flaws that match no rule, such as an insecure direct object reference (IDOR) or a business-logic error.

This is for you if you are choosing an AI SAST product or an AI triage layer for your current scanner. Skip this if you only need dependency or secrets scanning.

Key points

• AI SAST adds coverage where rules are weakest. Broken access control and security misconfiguration rank first and second in the OWASP Top 10:2025, and OWASP reports that 100% of the applications tested had some form of each.¹
• Rule-based SAST can be noisy. In a 2025 vendor report, three rules in one open-source engine produced 2,116 findings on 2,856 GitHub repositories, and 1,936 (91.5%) were false positives.² The figure covers those rules only, and Ghost Security, which wrote two of them, sells AI triage.

What is AI SAST, and how is AI-native different from AI-assisted?

AI SAST is static application security testing (SAST) in which a model owns one or more of four steps: detection, triage, fixing and validation. "Static" means the tool reads code without running it.

In AI-assisted tools, rule-based SAST (pattern or dataflow rules, as in Semgrep, CodeQL or SonarQube) produces the findings. The model may write rules for it, triage or explain findings, or draft fixes. In AI-native tools, the model reads the code and reports findings itself.

Agentic SAST is SAST in which the model works as an agent, at detection or triage, opening files and following calls over many steps. In Sifting the Noise (January 2026), four rule-based tools flagged 98.3% of the safe test cases in the OWASP Benchmark, a Java test application with planted vulnerabilities.³ A single prompt to Claude Sonnet 4 cut that false-positive rate to 23.0%; inside the SWE-agent framework, the same model reached 6.3%.

A finding is what the tool reports; a vulnerability is a real flaw in your code. Precision and recall measure the gap between them.

What does AI SAST find that rule-based SAST misses?

AI SAST finds flaws that have no fixed pattern, mainly missing authorization checks and business-logic errors. Rule-based SAST looks for known code patterns, or for untrusted input that reaches a dangerous function. An IDOR, where a handler returns a record without checking that it belongs to the caller, has no dangerous function to find. Semgrep, whose main product is a rule-based scanner, says such flaws need "complex custom rules" that few teams have time to write. As a result, "logic vulns go undetected in practice".⁴

A model can read the handler, the data model and the authorization code together and notice the missing check. Configuration works the same way: whether a public route or an exported Android component is a risk depends on the code behind it, and a model reads both.

Early results show the gain, with a lot of noise. In a September 2025 Semgrep study of 11 open-source Python web applications, IDOR was Claude Code's most accurate class. Of its 59 IDOR reports, 13 were real, against 2 of 38 for SQL injection.⁵

Where does the LLM sit in an AI SAST tool?

The model can sit at any of the four steps, and each position fails in its own way.

Measure precision after any stage where a model adds findings, and recall after any stage where it removes them.

Where does AI SAST fail?

Research shows two measured failures, and vendor figures leave two gaps.

Removing noise can remove vulnerabilities. The Sifting the Noise setup that reached 6.3% also marked 314 real vulnerabilities as false positives, out of 1,411 vulnerable test cases it completed.³

Results also vary between runs. In the Semgrep study, three identical runs on one application produced 3, 6 and then 11 distinct findings.⁵

The gaps are agreement and fix speed. A user-agreement rate for AI triage cannot show where the model and the reviewer were both wrong; only a labeled sample can. A time-to-fix figure does not show whether the vulnerability is gone.

How do you test a vendor's AI SAST claim?

Test on your own code, with labels you wrote before the vendor saw it:

• Pick two or three repositories in your main languages that the vendor has not seen.
• Label known vulnerabilities from past penetration tests and fixes, including access-control and business-logic flaws, plus findings your team closed as false positives, and record the sample size.
• Run the tool twice with the same settings and compare the findings.
• Count precision (real vulnerabilities among the findings, divided by all findings) and recall (labeled vulnerabilities found, divided by all labeled).
• Time triage per finding, and check a sample of the findings the AI dismissed.

Report precision and recall separately, because F1 hides which one moved.

Then ask the vendor in writing:

• Was your benchmark code, or ours, in the tool's training or tuning data?
• Can we see the raw output, including dismissed findings and the reasons?
• Which model runs each step, and does it change between releases?
• Where does inference run, and is our code stored or used for training?

Worked example: two models and triage memory in Haxset's SAST

Haxset's SAST scores agreement between models instead of trusting it. AI-Powered SAST and DAST describes the pipeline.

Each vulnerability class gets its own scan. Web classes include IDOR, authentication bypass, race conditions and business logic; mobile classes include exported components, debug and backup flags and client-side authorization. Full scans of web code also check infrastructure-as-code files for misconfiguration.

In thorough mode, two different models scan each class and a validator reviews their findings; fast mode uses one model and a final validator pass. The scorer starts from the model's confidence (1 to 10). Agreement adds 1, a validator's confirmation adds 1, and "needs investigation" subtracts 1. The scorer drops any finding a validator has marked false and filters out findings below 6.

A validator's notes on a finding from a Haxset scan of an open-source project, saying why the finding was kept.

A later step keeps severity within one step of each class's baseline, and rejected or filtered findings do not appear in the scan results.

Triage memory turns repeated human dismissals into a rule for one product across your organization. A weekly model pass groups dismissed findings. Code accepts a group only if it spans two or more scans and one vulnerability family (classes that share a CWE), with no approval step. The model's own dismissals never count. Matches show at info severity and keep their original severity, and admins can suspend, edit or delete rules.

The May post Security on Every Pull Request, and Every Week covers how these decisions carry across pull-request rescans.

How Haxset can help

The Haxset Engine runs this pipeline on web and mobile code in full scans, and a diff-based version of it on pull requests. Haxset does not keep the prompts its AI models process and does not train models on them. It stores only scan metadata and scan results. Haxset's operators also deliver manual penetration tests when you need a person to confirm what a scanner reports. To discuss either, contact us.

Related research

• AI-Powered SAST and DAST: How LLM Agents Find, Validate and Triage Vulnerabilities in Your Code and Running Apps
• Security on Every Pull Request, and Every Week: AI CI/CD Scanning and Continuous Monitoring

References

1. OWASP, OWASP Top 10:2025, categories A01:2025 Broken Access Control and A02:2025 Security Misconfiguration, 2025. ↩

2. Ghost Security, Exorcising the SAST Demons: How Ghost is Replacing Rule-Based Scanning with AI-Powered Triage and Detection, vendor report, April 2025. ↩

3. Y. Xiong, T. Zhang, Sifting the Noise: A Comparative Study of LLM Agents in Vulnerability False Positive Filtering, arXiv:2601.22952v1, 30 January 2026. ↩ ↩²

4. J. Moxon, Catching IDORs, Broken Authorization, and Other Logic Issues with Semgrep AI-Powered Detection, Semgrep, 11 November 2025. ↩

5. R. Gaucher, V. Ermilov, C. Gibler, Finding vulnerabilities in modern web apps using Claude Code and OpenAI Codex, Semgrep, 2 September 2025. ↩ ↩²