Why Most SAST Findings Are Noise, and How Multi-Model LLM Triage Changes the Math

TL;DR: In most published samples, most findings from rule-based static application security testing (SAST) are false positives, so measure your own noise on a labeled sample first. AI triage with a large language model (LLM) cuts noise but loses real findings. In a 2026 Tencent study of 433 mostly C/C++ bug findings, the best method removed 94–98% of false positives and kept 75–88% of real ones.¹ Haxset adds model agreement in thorough mode, a validator, a severity clamp and triage memory.

This is for you if your developers close most SAST findings without a fix, or a vendor is offering AI triage to cut them. Skip this if you want a ranked list of SAST tools; this post covers how to measure and reduce noise.

Key points

• Agreement is not accuracy. Semgrep Assistant agreed with Semgrep's researchers on 96% of true positives but only 41% of false positives (vendor figure, on its tuning set).²
• When real findings are rare, precision stays low after AI triage. In SastBench, 11% of findings were real, and the best setup raised that to 17% of the findings it kept.³
• In Haxset's SAST, a triage-memory rule needs human dismissals on at least two scans, and a finding it matches stays visible at info severity, with its original severity kept.

How noisy is SAST output?

In most published samples, fewer than a third of rule-based SAST findings were real. The samples differ, so read each row alone.

Help Net Security's write-up of the Ghost study gives the cause: a scanner "can't tell if the input is actually controlled by an attacker or if mitigating controls are in place".⁵

SonarQube is the exception, but its recall, the share of real vulnerabilities a tool finds, was 6.5%. RealVuln's authors make Kolega.Dev, a security-specialized scanner: 38.8% real, 80.9% recall. Tencent's developers spend "on average more than 10 minutes reviewing one single static bug alarm".¹

How do you measure SAST noise?

Label a random sample of your own recent findings, with someone who reads the code, and record two numbers. Precision is the share of findings that are technically real. The effective false-positive rate is the share closed without action. Google's static analysis team calls an issue an "effective false positive" if "developers did not take some positive action after seeing the issue".⁷ In this post, noise means that second number.

Agreement rates measure something else. Semgrep's researchers triaged more than 2,000 findings, and Semgrep chose its models and settings against that same set. Its AI matched them on 96% of the true positives and 41% of the false positives.² Semgrep calls the design "conservative": it rarely dismisses a real finding and leaves most false positives in place.

Precision also depends on how rare real findings are. In SastBench, a January 2026 benchmark for AI triage agents, 299 of 2,737 findings were real vulnerabilities, about 11%.³ The best setup reached a precision of 0.169, so 17% of the findings it kept were real. SastBench counts Semgrep findings that do not match a known vulnerability as false positives, and its authors note that some may be real.

How do I reduce false positives in SAST?

Reduce false positives in the rules and the scanner's context first, then check any AI triage you add:

1. Label a random sample of recent findings and record precision for each rule.
2. Rewrite or turn off the noisiest rules first, since the Ghost rules ranged from 0.5% to 20% real. A rule above 10% effective false positives fails the bar Google sets for new checks.⁷
3. Tell the scanner about your sanitizers, data access layer and framework protections, the controls a pattern cannot see.
4. Exclude test code and internal scripts, or label them so reviewers can skip them.
5. Record each dismissal with a reason, so the next scan does not raise it again.
6. Check what AI triage removes. If the tool does not list removed findings, run your current scanner on the same commit and label a sample of the findings only it reported.

Does AI triage reduce SAST false positives?

Yes, and in every study cited here it also dropped real findings. On Tencent's 433 findings (105 real bugs), the best method tested combines a model with static analysis. It removed 94–98% of false positives and kept 75–88% of real bugs, depending on the model.¹ The authors write that recall "remains below the enterprise-expected threshold of 90%". Our June post What Is AI SAST? covers a second study with the same trade-off.

Take 1,000 findings of which 100 are real, close to SastBench's 11%. Triage with Semgrep's reported rates keeps 96 real findings and 531 false positives: 627 to review, 15% of them real, with 4 real findings lost. One Tencent setup removed 98% of false positives and kept 75% of real findings: 93 to review, 81% of them real, with 25 lost. In these two examples, the triage that made the queue mostly real also lost the most real findings.

How Haxset's SAST pipeline removes noise

Haxset's AI SAST removes noise at several points, each aimed at one source. A model first profiles the codebase (frameworks, authentication model, entry points), and an agent for each vulnerability class reads the code with file and search tools. In thorough mode, the default, two different models scan each class.

The validator, a separate AI pass that can also read the code, then judges each finding. It checks whether an attacker controls the input, whether a sanitizer, the data access layer or middleware blocks the flow, and whether the code is a test or unreachable.

The scorer starts from the model's confidence (1 to 10), as revised by the validator. Agreement between the models adds 1, a validator's confirmation adds 1 and "needs investigation" subtracts 1. Agreement adds only one point, which fits SastBench's finding that models tend to be right or wrong together.

A finding the validator marks false is dropped, and any finding below 6 is dropped. A finding is settled when it is confirmed, found by both models and started at 8 or more; the others get a second validation pass. Fast mode uses one model and one validation pass.

A severity clamp then keeps each finding within one step of its class baseline, the usual severity for that class (medium for reflected XSS, critical for SQL injection).

The results page of a thorough-mode Haxset scan of OWASP Juice Shop: findings by severity, then each finding with its severity, location, CWE ID and status.

When AI triage hides a real finding

In Haxset's SAST, a real finding can be removed or lowered at these points:

• The validator marks it false, for example when it cannot find the vulnerable line itself.
• It scores below 6.
• The final review, a model pass over all findings, merges it into another finding.
• The clamp lowers a correct rating, such as a critical issue in a class whose baseline is medium.
• A triage-memory rule matches it.

Rejected findings and findings below 6 do not appear in the scan results. A data flow across many files can be real even when the validator fails to trace it. Two rules limit the validator. A claim the code supports but with no concrete attacker gain is kept and down-rated, not rejected; the clamp still holds it within one step of the class baseline. A settled finding skips the second validation pass, so that pass cannot remove it.

Triage memory works differently. A rule takes effect without an approval step. The weekly job, a model pass over your team's dismissals every Saturday, turns dismissals of one vulnerability family on two scans into a rule.

Suppose your team marks two SQL injection findings in an internal reporting module as accepted risk. Later findings in exactly that code are marked, and a model decides which other findings the rule covers. If it matches a new SQL injection in a public endpoint, that finding arrives as accepted risk at info severity.

Haxset marks such findings instead of removing them. Each carries an Auto-triaged badge and its original severity, the scan page counts them, and the completion email never reports a clean scan while any exist. Org admins can suspend, edit or delete a rule. Anyone on your team who triages findings can reopen a matched finding, which suspends the rule for the whole organization, and only an org admin can re-activate it. Only human decisions feed a rule, never the models' own dismissals.

Disclosure, and how Haxset can help

Haxset offers a product in this category. Haxset does not keep the prompts its AI models process and does not train models on them. It stores only scan metadata and scan results. To compare it with your current tool, we can scan one of your repositories and label a sample of both tools' findings with your team: contact us.

Related research

• What Is AI SAST? AI-Native vs AI-Assisted Static Analysis, and What the LLM Actually Does
• AI-Powered SAST and DAST: How LLM Agents Find & Validate Vulnerabilities in Your Code and Running Apps
• Security on Every Pull Request, and Every Week: AI CI/CD Scanning and Continuous Monitoring

References

1. X. Du et al. (Fudan University and Tencent), Reducing False Positives in Static Bug Detection with LLMs: An Empirical Study in Industry, arXiv:2601.18844v1, 26 January 2026. ↩ ↩² ↩³ ↩⁴

2. J. Moxon, S. Jaksik, How we built an AppSec AI that security researchers agree with 96% of the time, Semgrep, 22 January 2025. ↩ ↩²

3. J. Feiglin, G. Dar, SastBench: A Benchmark for Testing Agentic SAST Triage, arXiv:2601.02941v1, 6 January 2026. ↩ ↩²

4. Ghost Security, Exorcising the SAST Demons: How Ghost is Replacing Rule-Based Scanning with AI-Powered Triage and Detection, vendor report, April 2025. ↩

5. M. Zorz, 91% noise: A look at what's wrong with traditional SAST tools, Help Net Security, 19 June 2025. ↩ ↩²

6. J. Pellew, F. Raza, RealVuln: Benchmarking Rule-Based, General-Purpose LLM, and Security-Specialized Scanners on Real-World Code, arXiv:2604.13764v1, 15 April 2026. ↩

7. C. Sadowski, "Static Analysis", chapter 20 of Software Engineering at Google, O'Reilly, March 2020. ↩ ↩²