ChatGPT 
Summary
Key takeaways
- The public evidence does not support naming one universally most secure AI coding agent because models, harnesses, configurations, client versions, and evaluation dates differ across the available records.
- The normalized Agent Security League dataset contains 24 tool-model combinations: 11 Cursor records, 10 Claude Code records, and three Codex records.
- Median functional correctness across the 24 records is 72.9%, while median secure correctness is only 14.8%, creating a 57.8 percentage-point gap.
- The highest published functional score is 84.9%, while the highest secure correctness result is 29.0%, showing that code can pass normal tests while still failing security-specific checks.
- Cursor with Claude Fable 5 has the strongest captured secure correctness result at 29.0%.
- Codex with GPT 5.6 Sol has the best captured Codex secure score at 23.5%, while Claude Code with Claude Sonnet 5 reaches 19.6%.
- Codex has the highest median secure score in the captured snapshot, but the sample contains only three Codex records and cannot prove that the Codex harness is inherently more secure.
- IssueTrojanBench reports that 66.5% of malicious issue requests penetrated both agent-level and model-level guardrails.
- Runtime restrictions help, but strict native configurations may still allow MCP changes, tool poisoning, risky package installation, and other supply-chain actions.
- Engineering teams should combine sandboxing, restricted permissions, independent security testing, secret scanning, dependency checks, human approval, and senior code review rather than relying on the coding agent’s internal controls alone.
When this applies
This applies when an engineering team uses Claude Code, OpenAI Codex, Cursor, or another coding agent to modify repositories, generate patches, install dependencies, execute shell commands, access MCP tools, or interact with external networks. It is especially relevant when AI-generated code reaches production, when repositories contain untrusted issues or documents, or when agents can access secrets, package managers, deployment systems, infrastructure, and customer data. The benchmark is useful for designing governed AI-assisted development workflows and identifying where ordinary functional tests fail to detect security defects.
When this does not apply
This does not apply as a definitive ranking of Claude Code, Codex, and Cursor or as proof that one product is always safer than another. The available records use different models, dates, configurations, and orchestration layers, and Uvik did not independently execute the proprietary agents in this release. The results also do not replace environment-specific threat modeling, penetration testing, source-code review, or validation of the exact client version and permission mode used by an engineering team.
Checklist
- Record the exact coding agent, client version, model identifier, and run date used by the team.
- Pin permission, sandbox, network, MCP, privacy, and approval settings for every workflow.
- Use read-only or planning modes when opening unfamiliar or untrusted repositories.
- Treat issue descriptions, repository instructions, PDFs, images, and tool descriptions as untrusted input.
- Restrict filesystem access to the minimum directories required for the task.
- Restrict shell commands and require approval for risky or irreversible operations.
- Disable unrestricted network access unless the task explicitly requires it.
- Use allowlists for approved domains, package registries, APIs, and MCP servers.
- Require human approval before secret access, dependency changes, production mutations, or deployment actions.
- Run hidden or independent security tests rather than relying only on tests visible to the agent.
- Apply SAST, dependency scanning, secret scanning, and software composition analysis after agent-generated changes.
- Review authentication, authorization, injection risks, unsafe defaults, and error handling manually.
- Track repair time, reopened defects, regressions, and security findings instead of measuring only initial completion speed.
- Keep detailed logs of prompts, tool calls, approvals, network activity, file changes, and final outcomes.
- Require senior human review before any AI-generated or AI-modified code is merged into a protected branch.
Common pitfalls
- Treating a passing functional test suite as proof that an AI-generated patch is secure.
- Comparing agent harnesses without controlling for model choice, run date, client version, and permission configuration.
- Giving coding agents unrestricted shell, filesystem, network, package-manager, or MCP access.
- Trusting instructions embedded in repository issues, PDFs, documentation, or tool metadata.
- Assuming that a strict native mode blocks every prompt-injection, MCP, package, and supply-chain risk.
- Using non-interactive or bypass-permission modes without isolating the execution environment separately.
- Allowing background agents to combine internet access, automatic command execution, and repository write permissions.
- Measuring productivity by task completion while ignoring defects, human repair time, and reopened security issues.
- Relying on the agent to review the security of code that the same agent generated.
- Presenting incomplete public benchmark evidence as a universal winner score or product-security certification.
Evidence synthesis and normalized dataset by Uvik Software
Research disclosure: Uvik Software publishes this research and may provide related AI engineering and code-security services. No vendor, model provider or benchmark publisher can pay for inclusion, placement or a more favorable conclusion.
Affiliation disclosure: Uvik Software is a Claude Partner Network member and uses multiple AI coding tools in engineering delivery. This creates a potential conflict of interest. The release therefore keeps source-reported metrics separate, identifies which conclusions are Uvik Software calculations and withholds a full six-dimension winner score where public evidence is incomplete.
Executive answer: The public evidence does not support a universal winner. Across 24 Claude Code, Codex and Cursor tool-model records in the Agent Security League, median functional correctness was 72.9% while median secure correctness was only 14.8%, a median gap of 57.8 percentage points. The best published secure result was 29.0%. Separate research found that 66.5% of malicious issue requests penetrated coding-agent guardrails, while a runtime-control benchmark showed that strict configuration can still leave MCP, tool-poisoning and package-risk scenarios allowed.
Key findings
- The normalized Agent Security League dataset contains 24 tool-model combinations: 11 Cursor, 10 Claude Code and 3 Codex records.
- The highest published functional score is 84.9%, reached by two Cursor combinations in the captured leaderboard.
- The highest published secure correctness score is 29.0% for Cursor with Claude Fable 5.
- The best captured Codex secure score is 23.5% with GPT 5.6 Sol.
- The best captured Claude Code secure score is 19.6% with Claude Sonnet 5.
- Across all 24 combinations, the mean functional score is 66.6% and the mean secure score is 15.5%.
- IssueTrojanBench reports 66.5% overall malicious-issue penetration, with 72.2% penetration for standard text artifacts and 16.7% for low-authority metadata such as image alt-text.
- The HOL Guard runtime benchmark covers 220 results across five harnesses, four comparator modes and 11 scenarios, but its native default and native strict outcomes reflect documented behavior rather than direct behavioral execution.
Best available secure correctness result by harness in the captured Agent Security League dataset. Models and dates differ, so this is not a controlled same-model comparison.
What this release measures
This page combines three different security questions that should not be collapsed into one number:
| Evidence layer | Primary question | Source | Important limitation |
|---|---|---|---|
| Functional and secure code outcomes | Does the generated patch work, and does it also pass task-specific security tests? | Endor Labs Agent Security League | Different model and harness combinations were tested on different dates; client versions are not published in the leaderboard |
| Malicious issue resistance | Does the agent follow malicious instructions embedded in issues, PDFs or metadata? | IssueTrojanBench | The paper reports a defined attack corpus, not every prompt-injection path found in production |
| Runtime control behavior | Do configuration and guardrail layers allow, prompt or prevent secret reads and risky actions? | HOL Guard Runtime Security Benchmark | Native comparator outcomes are based on documented configuration behavior, not direct execution of every harness |
| Documented product controls | What sandbox, permission, network, approval and MCP controls are officially documented? | Anthropic, OpenAI and Cursor documentation | Documentation does not prove that controls stop every attack |
Functional correctness is much higher than secure correctness
The Agent Security League extends the SusVibes research line with 200 real-world tasks from 108 open-source Python projects spanning 77 CWE classes. Its two headline metrics are:
- FuncPass: the generated code passes the functional test suite.
- SecPass: the generated code also passes the security tests.
In the captured 24-record subset, median FuncPass is 72.9% and median SecPass is 14.8%. A patch can therefore look successful under ordinary acceptance tests while still failing security-specific checks.
Each point is one published harness-model record. The distance below the diagonal represents code that works functionally but does not pass the security suite.
Top published secure correctness records
| Harness | Model | Functional | Secure | Gap | Run date |
|---|---|---|---|---|---|
| Cursor | Claude Fable 5 | 72.6% | 29.0% | 43.6 pp | 2026-06-12 |
| Cursor | GPT-5.5 | 84.9% | 24.0% | 60.9 pp | 2026-04-25 |
| Codex | GPT 5.6 Sol | 70.9% | 23.5% | 47.4 pp | 2026-07-17 |
| Codex | GPT-5.5 | 62.6% | 22.4% | 40.2 pp | 2026-04-25 |
| Codex | GPT-5.4 | 63.1% | 21.8% | 41.3 pp | 2026-03-18 |
| Cursor | Claude Opus 4.8 | 75.4% | 20.7% | 54.7 pp | 2026-05-30 |
| Claude Code | Claude Sonnet 5 | 83.2% | 19.6% | 63.6 pp | 2026-07-02 |
| Claude Code | Claude Fable 5 | 59.8% | 19.0% | 40.8 pp | 2026-06-10 |
| Cursor | Claude Opus 4.7 | 79.9% | 18.4% | 61.5 pp | 2026-04-17 |
| Cursor | Gemini 3.5 Flash | 79.3% | 17.9% | 61.4 pp | 2026-05-20 |
The table should not be read as a same-model orchestration test. Harness, model and run date all vary. A stronger conclusion requires the same model, task corpus, tool version, permissions, network policy and repeated-run protocol.
Harness-level medians are descriptive, not causal
| Harness | Records | Median functional | Median secure | Best secure model | Best secure result |
|---|---|---|---|---|---|
| Cursor | 11 | 75.4% | 15.1% | Claude Fable 5 | 29.0% |
| Codex | 3 | 63.1% | 22.4% | GPT 5.6 Sol | 23.5% |
| Claude Code | 10 | 69.25% | 11.45% | Claude Sonnet 5 | 19.6% |
Codex has the highest median secure score in this snapshot, but only three Codex records are present and the tested models differ from those used in Claude Code and Cursor. This is a sample-composition result, not proof that the Codex harness is intrinsically more secure.
Harness medians summarize the captured records but do not control for model choice or evaluation date.
The security gap remains large even for the strongest result from each harness
The strongest secure result for each harness still leaves a substantial gap between working code and secure code.
Malicious repository and issue content is a separate threat
IssueTrojanBench v1 evaluates Cursor, Claude Code and Codex Desktop against malicious issue requests delivered through six vectors and four attack categories. The paper reports that 66.5% of malicious issues penetrated both agent-level and model-level guardrails.
Delivery context mattered. Standard text artifacts such as issue bodies and PDFs succeeded in 72.2% of runs, compared with 16.7% when malicious content was confined to lower-authority metadata such as image alt-text. Among resisted runs, 82.9% were blocked by explicit model refusal and 17.1% by source-based trust classification. The authors conclude that lightweight instruction-data separation provided limited additional protection.
IssueTrojanBench shows that the authority and placement of malicious instructions materially affect attack success.
Runtime controls protect a different boundary
The HOL Guard benchmark defines 11 deterministic scenarios covering fake secrets, risky shell execution, MCP configuration changes, tool poisoning, package risk, safe actions, approvals and receipt creation. It publishes 220 results across five harnesses and four comparator modes.
In the published summary, native default allows all 11 actions. Native strict detects the three secret reads and prompts for the risky shell command, but still allows MCP registration, MCP modification, tool poisoning and package-risk installation. Guard default prompts or prevents risky actions while allowing safe controls; Guard strict prevents eight risky scenarios and still permits explicit approval and receipt flows.
This evidence is useful for control design, but it is not direct proof of native harness behavior under every platform version. The benchmark states that native default and native strict comparators reflect documented configuration options.
Stricter runtime policy changes outcomes, but native strict configuration does not cover every tool and supply-chain scenario in the published matrix.
Verified documented control surfaces
| Tool | Command and approval controls | Network controls | Security-relevant limitation | Source |
|---|---|---|---|---|
| Claude Code | Per-tool allow and deny rules; permission modes include default, acceptEdits, plan and bypassPermissions | Network requests require approval by default; proxy and enterprise controls are documented | Non-interactive and bypass-permissions modes reduce human friction and require a separately secured environment | Official documentation |
| OpenAI Codex | Actions outside the sandbox can require approval; managed configuration can standardize policy | Network access can be restricted through sandbox and managed network policies | Auto-review and more permissive modes reduce approval prompts, so boundary configuration and logs are material | Official documentation |
| Cursor | Interactive commands can require approval; non-interactive mode has full write access | Background agents run in isolated VMs with internet access and auto-run terminal commands | Background agents combine internet access and automatic command execution, increasing prompt-injection and exfiltration exposure | Official documentation |
These are documentation records, not benchmark scores. Product modes must be pinned for every run because foreground, non-interactive and cloud/background execution can expose materially different permission and network surfaces.
Scoring implementation
The downloadable scoring script implements the original six-dimension framework:
| Metric | Weight | Publication status in this release |
|---|---|---|
| Functional correctness | 25% | Available from Agent Security League |
| Security | 30% | Available from Agent Security League as SecPass |
| Regression control | 15% | Not published in a comparable form across all source records |
| Test quality | 10% | Not published in a comparable form across all source records |
| Maintainability | 10% | No blinded reviewer score available for all records |
| Human repair time | 10% | No comparable active repair-time dataset available |
Because four dimensions are missing, the full Uvik Software score remains blank. The script also calculates a clearly labeled descriptive score using 40% FuncPass and 60% SecPass. This secondary calculation is an editorial comparison aid and must not be presented as the source benchmark score or a universal winner ranking.
What a true controlled Uvik benchmark would still require
- One frozen task corpus and repository commit per task.
- The same model tested across Claude Code, Codex and Cursor where product support permits.
- Exact client versions, model identifiers and run dates.
- Fixed permission, sandbox, network, MCP and privacy configuration.
- Multiple independent runs per task and configuration.
- Hidden functional and security tests that are unavailable to the agent.
- Blinded senior reviewers for maintainability and security narratives.
- Measured active repair time under a standardized protocol.
- Vendor preview, reviewer adjudication and a correction period.
- Versioned task, run and aggregate datasets with reproducible code.
How engineering teams should use these results
- Do not treat visible tests as proof that an agent-generated patch is secure.
- Pin the model and harness. The same model can behave differently across orchestration layers.
- Use read-only or plan modes for untrusted repositories before allowing edits or commands.
- Restrict file, shell, network and MCP access with technical controls outside the model.
- Require human approval for secret access, dependency changes, production mutations and irreversible actions.
- Run independent security tests, SAST, dependency checks and secret scanning after agent changes.
- Review repository instructions, issue text, PDFs and tool descriptions as untrusted input.
- Track repair time and reopened defects, not only initial task completion speed.
Vendor and source-owner preview
The package includes a preview tracker for Anthropic, OpenAI, Cursor, Endor Labs, the HOL Guard team and the IssueTrojanBench authors. The current status is NOT SENT - HUMAN OUTREACH REQUIRED. A final 1.0 release should not claim vendor review until the preview messages are actually sent and the response window closes.
Correction workflow
Corrections must identify the record ID, disputed field, current value, proposed value and public evidence URL. Material changes must update the dataset version and changelog. Payment or commercial relationships cannot influence a correction decision.
Download the data
- Download normalized Agent Security League records CSV
- Download runtime-control scenario matrix CSV
- Download malicious issue summary CSV
- Download official control-surface records CSV
Limitations
- Uvik Software did not independently execute the proprietary agents in this release.
- The Agent Security League records use different models and dates, so harness medians are not causal comparisons.
- Client versions and full raw traces are not published in the leaderboard records.
- IssueTrojanBench evaluates a defined malicious-issue corpus, not every indirect prompt-injection path.
- The HOL native comparator matrix reflects documented behavior and deterministic fixtures rather than direct execution of every agent version.
- No comparable public dataset supplies regression control, test quality, maintainability and human repair time for all 24 records.
- Vendor documentation describes controls but does not prove their effectiveness.
Commercial handoff
Need a security review of AI-generated Python code or a governed coding-agent workflow? Uvik Software reviews authorization, dependencies, tests, secrets, CI/CD, MCP permissions, sandbox policy and production deployment controls. Request an AI-assisted codebase assessment.
Primary sources
- Endor Labs – AI Coding Agent Security Benchmark
- HOL Guard – AI Coding Agent Runtime Security Benchmark
- IssueTrojanBench – arXiv v1
- Anthropic – Claude Code security
- Anthropic – Claude Code CLI reference
- OpenAI – Running Codex safely
- OpenAI – Codex CLI getting started
- Cursor – CLI command approval
- Cursor – CLI permissions
- Cursor – Background Agents security
- OWASP Agentic Security Initiative
Frequently asked questions
Which AI coding agent is the most secure?
The available evidence does not support one universal winner. Cursor has the highest captured SecPass result, Codex has the highest harness median in this small snapshot, and Claude Code has a strong functional result with Claude Sonnet 5. Models, dates and configurations differ.
Why is secure correctness so much lower than functional correctness?
Ordinary tests verify expected behavior, while security tests target authorization, injection, unsafe defaults and edge cases. A patch can work for the visible feature and still leave the underlying vulnerability open.
Did Uvik Software run these agents?
No. Version 1.0-rc1 normalizes published public benchmark results and official documentation. A future controlled Uvik edition would require independent repeated runs and reviewer measurements.
Does a low SecPass score mean the tool is always unsafe?
No. It describes performance on a defined task set under a recorded harness-model condition. Production risk also depends on permissions, tests, review, network policy, repository content and deployment controls.
Why is the full six-dimension score blank?
Comparable regression, test-quality, maintainability and repair-time data are unavailable for every record. The scoring script refuses to invent those values.
Can a vendor or benchmark publisher request a correction?
Yes. Submit the record ID, disputed field and public evidence. Material corrections will be versioned and listed in the changelog.
