Security on Every Pull Request, and Every Week: AI CI/CD Scanning and Continuous Monitoring

TL;DR: Haxset scans each pull request when it is opened or reopened and posts one comment labeling findings F# (code), S# (secrets), I# (misconfigurations) and D# (dependencies). On GitHub, developers reply with /haxset fix, fp or accept. Between releases, full SAST and DAST run weekly, monthly or quarterly, and rescans confirm that fixes hold.

This is for you if your developers ship code fast or utilize AI coding to help deliver faster. Also, if your auditor asks how each change was security-checked. Skip this if your code doesn't change frequently or is not hosted in your environment.

Key points

• By default, an AI code finding is run through a validator model, which will ensure exploitability in the code.
• A one-click fix is offered only for applicable findings.
• Two human dismissals of the same kind of weakness on two scans become an organization-wide rule, with no approval step.

Why a pull-request scan finds problems a quarterly scan reports late

A pull-request scan flags the lines a developer just changed, while the author still has the context. A quarterly scan reports it up to three months after the merge. Veracode's 2026 State of Software Security report, drawn from 1.6 million applications, found that 82 percent of organizations carry security debt.¹ Its 2025 report put the average time to fix a flaw at 252 days.² Neither figure measures pull-request scanning, but both show the queue a late finding joins.

How do you add security scanning to GitHub Actions and GitLab CI?

Both start from a CI token generated in the Haxset dashboard under CI/CD. The runner uploads the diff and the changed files to Haxset's API at app.haxset.com, where the scan runs, so it needs outbound HTTPS access. Unchanged files and .git stay on the runner.

GitHub Actions

Save the token as the secret HAXSET_SCANNER_TOKEN and add a workflow like this one, adapted from the action's README:³

hljs yaml
on:
  pull_request: { types: [opened, reopened] }
  issue_comment: { types: [created] }
permissions: { contents: read, pull-requests: write }
jobs:
  haxset:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0
          ref: ${{ github.event_name == 'issue_comment' && format('refs/pull/{0}/head', github.event.issue.number) || '' }}
      - uses: haxset/security-scanner-action@v1
        with:
          token: ${{ secrets.HAXSET_SCANNER_TOKEN }}

fetch-depth: 0 lets the scanner diff against the base commit. The ref: line makes a comment run check out the pull request, not the default branch. Reviewers commit suggested fixes themselves, so the action never pushes code. Pin a commit SHA instead of the moving @v1 tag if you want to choose when to upgrade.³

GitLab CI

Add two Masked variables with Protected off. The first, HAXSET_SCANNER_TOKEN, holds the CI token; the second, HAXSET_GITLAB_TOKEN, is a GitLab access token with the api scope and Reporter role.⁴ Protected variables reach merge request pipelines only when both branches are protected and a project setting allows it.⁵ Unprotected, both tokens are readable by any pipeline in the project, and masking hides them only in job logs.⁶ Use a project access token, as the setup guide does, give it an expiry date, and rotate both tokens when someone with push access leaves.

Commit the template next to your .gitlab-ci.yml and include it:

hljs yaml
workflow:
  rules:
    - if: $CI_PIPELINE_SOURCE == "merge_request_event"
    - if: $CI_COMMIT_BRANCH

include:
  - local: haxset-security-scanner.gitlab-ci.yml

The workflow: block must sit in .gitlab-ci.yml itself. Rules that arrive through include: don't create merge request pipelines.⁵ The template adds an automatic scan job and manual jobs to rescan, re-check fixes and scan the whole repository.⁴ GitLab has no comment-triggered pipelines, so you triage in Haxset from the link in each note.

What happens when Haxset scans a pull request

The scan runs deterministic and AI scanners on the uploaded files:

• Changed dependency manifests are checked against public vulnerability databases, added lines are checked for secrets, and infrastructure-as-code (IaC) files are checked for misconfigurations.
• By default, a primary model and a faster second model scan the source files in parallel, follow data flow across them and score each finding from 1 to 10.
• IaC results, exact-format secrets and findings both models scored 16 or more in total (by default) skip the validator.
• The validator, a separate AI pass, re-reads the code for every other candidate and drops those it judges to be false positives. We don't publish its false-positive rate.

Each code finding in the comment carries a CWE ID, and the comment names any changed file the runner didn't upload, such as a symbolic link. Our May 12 post, AI-Powered SAST and DAST, describes the full pipeline this diff scan is based on.

The summary comment on a demo pull request, with findings shown. Developers use each finding's label, such as F1, in /haxset commands.

The /haxset commands developers use on a pull request

On GitHub, developers answer the scan with a comment. The action takes commands from the repository owner, organization members and collaborators, ignores bots, and does not check repository permissions.³

Each scan also writes fixes for findings on changed lines; on GitLab, these are the only fixes. A fix must pass guards such as a byte-for-byte match of the lines it replaces. By default, it is offered only if a scan of the patched copy shows the finding gone, and nothing is committed until a reviewer clicks.³ Secrets and vulnerable dependencies get no fixes.

A verified fix posted as a GitHub suggestion. The branch does not change until a reviewer clicks Commit suggestion.

Rescans that remember triage decisions

On a pull request, a dismissal is tied to the finding's fingerprint, so it survives line moves and stays hidden on the next scan. Each decision records who made it and when.

A repeat /haxset fullscan re-checks earlier findings and searches only files changed since, unless more than 400 changed. Dismissals carry over with their justifications, and new findings are labeled "New in this run".

Triage memory works across the organization. Every Saturday at 05:00 Riyadh time, an AI judge groups human dismissals by weakness. Dismissals of one weakness family on two scans become a rule for that product, even if one person made both. Matching findings get the rule's status at info severity, with the real severity kept, and admins can suspend or delete rules.

What does continuous security monitoring look like between releases?

Between releases, monitoring runs on a schedule, with SAST and DAST repeating weekly, monthly or quarterly. A failed scheduled scan emails its owner. After three failures in a row the schedule turns itself off silently, so check the schedule list when you collect evidence.

Rescans verify fixes. A SAST rescan marks a finding fixed only when the AI quotes the new code and the quote checks out. A DAST rescan marks it fixed only when the re-sent test reached the endpoint, got a real answer rather than a firewall block, and did not reproduce the flaw.

Using scan records as evidence for ECC 1-6 and SAMA CSF 3.3.6

Neither NCA's Essential Cybersecurity Controls (ECC) nor SAMA's Cyber Security Framework (CSF) mentions pull requests, but both ask for security checks on software changes. The mapping below is ours; neither regulator has reviewed it.

ECC-2:2024 applies to government entities and private operators of critical national infrastructure.⁷ Control 1-6-2 reads:

"The cybersecurity requirements for project management and information and technology asset changes within the entity shall include the following as a minimum: 1.6.2.1 Vulnerability assessment and remediation."

SAMA CSF Version 1.0 (May 2017) applies to all Member Organizations SAMA regulates, including banks, insurers and financing companies.⁸ Its change-management control, 3.3.7, lists under 4.b:

"security testing, which should (if applicable) include: 1. penetration testing; 2. code review if applications are developed internally;"

Quoted text in the table is the control's wording; the rest is our paraphrase.

The automatic record shows the code when the pull request opened, so run /haxset scan before merge or rely on scheduled whole-repository scans. Before your next review, check that every repository in scope has a scan history.

Limits to know before you rely on it

The automatic scan runs once, when a pull request opens or reopens. Findings never block a merge: the GitHub job always finishes green and GitLab jobs allow failure.³ We chose that so an outage on our side never stops your merges. A pull-request scan can also miss a flaw rooted in unchanged code.

How Haxset can help

These are capabilities of the Haxset Engine. Haxset does not keep the prompts its AI models process and does not train models on them. It stores only scan metadata and scan results. To try the scanner on your repositories, contact us.

FAQ

Does it scan pull requests from forks?

Not automatically. GitHub gives a fork's workflow run no repository secrets and a read-only token, so it can neither scan nor comment.⁹ A user the action accepts can comment /haxset scan instead; that run uses the base repository's secret and checks out the fork's code. It never executes that code, but git does apply the pull request's own .gitattributes when building the diff. On GitLab, a fork's pipeline runs in the fork without your variables, so it isn't scanned; if a member runs it in your project, it uses the fork's CI configuration with your variables.⁵

Can a developer dismiss findings on their own pull request?

Yes, on GitHub, if the action accepts their comments. Each decision records who made it, so reviewers can check.

Does it work on self-managed GitLab or on runners without internet access?

The runner needs outbound HTTPS access to app.haxset.com. The GitLab template reaches GitLab through the API address each job receives (CI_API_V4_URL), so it is not tied to gitlab.com.

Does a pull-request scan replace a full SAST scan or a penetration test?

No. It reads only the changed files, not the code they call or the running application. Run /haxset fullscan for the repository, schedule DAST for the application, and keep your penetration tests.

Related research

• AI-Powered SAST and DAST: How LLM Agents Find, Validate and Triage Vulnerabilities in Your Code and Running Apps

References

1. Veracode, Veracode 2026 State of Software Security Report Reveals Four Out of Five Organizations Are Drowning in Security Debt, press release, February 24, 2026. ↩

2. Veracode, Veracode Reveals Half of Organizations Burdened by Critical Security Debt, press release on the 2025 State of Software Security report, February 27, 2025. ↩

3. Haxset, Security Scanner GitHub Action README, GitHub repository. ↩ ↩² ↩³ ↩⁴ ↩⁵

4. Haxset, GitLab CI template haxset-security-scanner.gitlab-ci.yml, CI template. ↩ ↩²

5. GitLab, Merge request pipelines, GitLab Docs: prerequisites, access to protected variables, and use with forked projects. ↩ ↩² ↩³

6. GitLab, CI/CD variables, GitLab Docs: protect variable and mask a CI/CD variable. ↩

7. National Cybersecurity Authority (NCA), Essential Cybersecurity Controls (ECC-2:2024), English edition, controls 1-6-2, 1-6-3 and 2-10-3. ↩

8. Saudi Arabian Monetary Authority (SAMA), Cyber Security Framework, Version 1.0, May 2017, controls 3.3.6, 3.3.7 and 3.3.17. ↩

9. GitHub, Events that trigger workflows, GitHub Docs, section on pull_request workflows in forked repositories. ↩