一句话总结
When Google released Gemini 4 Carbon on October 28, 2026, it disclosed scores on SEC-bench Pro, ExploitGym and CWE-bench, cementing cybersecurity as a core evaluation dimension for frontier models; this article explains what each benchmark tests, how it works, and how to read the numbers.
Background: Why Cybersecurity Became a New Evaluation Focus
Since 2026, frontier model vendors have begun publishing results for vulnerability discovery, exploitation and patching. On October 28, 2026, Google disclosed Gemini 4 Carbon's cybersecurity results alongside the release: 88.2% on SEC-bench Pro (vulnerability discovery), 54.8% on ExploitGym (exploitation) and 71.0% on CWE-bench (patching). In the same data set, Claude Opus 5.5 scored 34.5% on ExploitGym and 67.0% on CWE-bench, while GPT-6 Astra scored 85.4% on SEC-bench Pro, 42.4% on ExploitGym and 68.0% on CWE-bench (per MarkTechPost). Cybersecurity has moved from a bonus metric to a required one.
SEC-bench Pro: How Vulnerability Discovery Is Tested
SEC-bench Pro is an upgrade of SEC-bench, introduced in 2025 with contributions from Meta and others; it evaluates a model's ability to locate security vulnerabilities in codebases. A typical task provides a real or synthetic software repository and asks the model to determine whether a vulnerability exists, where it is (file and line), and what defect class it belongs to. Scoring is mainly based on localization accuracy. Gemini 4 Carbon leads with 88.2%, ahead of GPT-6 Astra's 85.4%; Claude Opus 5.5 has not disclosed a result. This benchmark fits code-audit-style tasks, the most common need in security operations.
ExploitGym: How Exploitation Is Tested
ExploitGym is driven by organizations including the US Defense Advanced Research Projects Agency (DARPA); it evaluates whether a model can turn a known vulnerability into a working exploit in a controlled environment. It simulates an isolated target machine where the model must write or invoke code to trigger the vulnerability and satisfy preset success conditions; automated judging reduces human grading bias. Because exploitation capability is directly linked to attack risk, scores tend to be lower than discovery benchmarks: Gemini 4 Carbon at 54.8%, GPT-6 Astra at 42.4%, and Claude Opus 5.5 at 34.5%. Lower scores reflect both task difficulty and vendor choices around safety guardrails.
CWE-bench: How Patching Is Tested
CWE-bench uses the Common Weakness Enumeration (CWE) framework to evaluate whether a model can generate correct patches for located vulnerabilities. Tasks typically provide vulnerable code plus context and require a minimal, compilable patch that preserves original functionality, verified through regression tests. Gemini 4 Carbon scores 71.0%, Claude Opus 5.5 67.0%, and GPT-6 Astra 68.0%. Patch benchmarks are closest to everyday development and security operations, demanding code quality and engineering-constraint understanding.
First, check the environment: the figures above are measured in an unrestricted test setting and do not represent default model behavior. Second, check the source: SEC-bench Pro and CWE-bench are maintained by third parties, and ExploitGym is an independent environment; always confirm the benchmark version and test date. Third, separate capability from safety: a low exploitation score does not mean a model is safer, it means the task is hard, and frontier models typically apply stricter output restrictions on exploit-type tasks. Fourth, weigh cost and access: Google initially grants Gemini 4 Carbon only to vetted cyber defenders via the Fairwind program, so a high score does not equal broad availability.