Orchestrate security scans with AI agent review and fix prep

Combining security scanners with the Agents SDK

In this notebook, we'll build a security review workflow with the OpenAI Agents SDK and static scans from Semgrep and Bandit. The manager agent in our workflow dynamically chooses specialist agents to investigate the code behind each scanner finding. After review and validation, we'll use the confirmed findings to create a Codex /goal prompt that you can review before asking Codex to make fixes.

Contents

1. Requirements and approvals
2. How the review works
3. Pin source files and run scanners
4. One manager chooses specialists
5. Validate findings and prepare a Codex goal
6. Compare two repository reviews

1. Requirements and approvals

• Python: Use Python 3.12.

• API key: Live runs require OPENAI_API_KEY.

• Supported platforms: Live scans support macOS, Linux, and Windows Subsystem for Linux (WSL). Native Windows is unsupported.

• JupyterLab: Install it separately.

• Packages and scanners: Install the pinned versions below.

  python -m pip install openai-agents==0.22.0 openai==3.6.0 pydantic==2.13.5 ipython==9.16.1 ipykernel==7.3.0
  python -m pip install semgrep==1.174.0 bandit==1.9.4
  

• Default model: gpt-5.6-sol with xhigh reasoning.

• Approvals: network allows downloads, scanners allows local scans, and model_agents allows model calls.

• Safe inspection: Leave approvals empty to run all cells without external actions.

  SECURITY_SWARM_APPROVALS="" jupyter lab examples/agents_sdk/security_scanners_with_agents_sdk.ipynb
  

• Full demonstration: Enable all three approvals.

  OPENAI_MODEL="gpt-5.6-sol" OPENAI_REASONING_EFFORT="xhigh" \
  SECURITY_SWARM_APPROVALS="network,scanners,model_agents" \
  jupyter lab examples/agents_sdk/security_scanners_with_agents_sdk.ipynb
  

Load the helper module

security_review_helpers.py handles downloads, scanner execution, evidence checks, and result formatting. Keep the helper in the same directory as this notebook if you are running it locally. The agent definitions, validator, and results are below.

2. How the review works

Now let's take a look at how this works. We start by collecting possible issues from the scanner reports into a list of candidates. The manager agent chooses a specialist to review each candidate, and an independent validator proposes a decision. Python checks that every candidate was reviewed exactly once and stops the review if a candidate ID is unknown, duplicated, or missing.

Python also enforces approvals, verifies files, and selects scanners. Each specialist agent receives excerpts from the verified files. We instruct the agents to treat repository text as evidence, never as instructions.

Both examples use static source analysis. They don't invoke Codex Security, run either application, or modify the source. You could build on this workflow by adding runtime checks or applying fixes, with human approval before either step.

3. Pin source files and run scanners

Now, let's get into a live run. We'll use a few files from two intentionally vulnerable GitHub projects. SasanLabs VulnerableApp gives us one Java/JDBC class, and OWASP crAPI gives us three Python files and one JavaScript file.

In this workflow, Semgrep uses five local rules to scan the selected files from both projects, and Bandit checks the production Python files. Complete coverage means those files were scanned, not that every type of vulnerability was checked. Findings must reference files the scanners actually inspected. If a scan fails or coverage is incomplete, the review will automatically stop.

We check each scanner result against the approved source and record the file path, line number, file hash, and commit, along with the scanner and rule ID. If multiple reports flag the same category of issue at the same code location, we group them into one candidate.

4. One manager agent chooses specialists

The manager can call authentication, injection, or configuration specialists. We give it the scanner candidates and the repository’s languages and source paths, then let it decide which specialist should review each candidate. Each selected specialist gets one batch of candidates. Agent.as_tool() lets the manager call a specialist without handing over the conversation.

Before a specialist starts, we check the candidate IDs and send it the relevant code from the verified files. Once its response passes the checks we save the assessment for the validator agent.

To extend this workflow, you could add specialists for API authorization, cryptography, sensitive data exposure, or cloud configuration.

5. Validate findings and prepare a Codex goal

Once the specialists finish their reviews, an independent validator proposes confirmed, needs_review, or not_actionable for each candidate, with a reason and any proof gaps. Python checks the candidate IDs in each proposal and maps them back to the verified source and scanner records.

A proof gap in this workflow is a missing fact that could change our judgment about the code we're reviewing. Take the JWT candidate in the saved run. The code skipped local signature verification, but it also called another service to verify the token. That service’s implementation wasn’t in our approved files, so the candidate stayed in needs_review.

Not every unknown is a proof gap, though. Questions about deployment or exposure go under limitations if they don’t change what we can conclude from the source. A finding can only be confirmed when it is tied to verified source and has no unresolved proof gaps.

Before building the /goal prompt, we check the evidence one last time (source hashes, line numbers, and scanner coverage). Invalid evidence stops the run. Python keeps any proof gaps from either review, even if the validator disagrees. Those findings stay in needs_review and out of the prompt. Only confirmed findings make it into the final /goal prompt.

6. Compare two repository reviews

Let’s run both repositories through the same run_review() function and see which specialists the manager chooses for each. When all three approvals are enabled, the final cell checks that both reviews completed and that we have a Codex /goal prompt to review.

The specialist column shows which specialist the manager called, and final shows the decision after the evidence checks. Notice the saved JWT result. It stays needs_review because the upstream verifier isn't part of the approved source. We'd need to inspect that verifier or show a bypass before making a stronger claim.

The crAPI SQL candidate needs more evidence too. We can see the serializer check, but not its validation rules.

The source links open the pinned files at the reported lines. Full reasons and proof gaps follow the tables. On small screens, scroll the tables horizontally.

Codex handoff for human review

We pick the completed review with the most confirmed findings and turn those findings into a /goal prompt for Codex. Python does the formatting to avoid additional model calls. In under 4,000 characters, the prompt tells Codex which verified files to work on and asks for focused regression tests.

Usage and limits

The table below shows how many model responses and tokens each stage used.

This example covers five source files. It doesn't cover runtime behavior, deployments, dependencies, infrastructure, secrets, or the rest of either repository. You'd need separate evaluation and human review before using this in production.

Conclusion

We used the same workflow to review two projects, with the manager choosing which specialists to call. In the saved run, ten candidates produced six confirmed findings, two marked not_actionable, and two left in needs_review. The four confirmed crAPI findings became a Codex /goal prompt we can review before requesting fixes.

If you adapt this to your own code, start with a small, approved set of files you know well. Check the manager’s assignments and the final decisions against your own review. Once you understand where it gets things right (and where it needs your help/additional context) you can decide what to add next.