Static leads and a verified RCE
Granite’s recorded investigations produced static candidate inventories and code-level evidence. Our team subsequently verified one RCE dynamically using a required test harness. These are distinct outcomes. [1] [2]
Dynamically verified by our team
1RCE dynamically verified
Required test harness
Our team dynamically verified one remote code execution finding under the harness setup. Other findings remain static candidates. [2]
Recorded separately from the static inventories, without target or workflow attribution.
Read the verification scopeStatic candidates · Target A
19static research leads
Granite Desktop’s IDs form 18 provisional root-cause families and include conditional items. They are not a verified-vulnerability count. [1]
Examine Target A’s findingsStatic candidates · Target B
Static candidates across both Granite workflows
- Granite Desktop
- 16
- Granite MCP
- 13
Reported static out-of-bounds candidates from separate workflows. The inventories are not added together. [1]
Examine Target B’s findingsThese are seven workflow arms across two datasets, not seven independent target trials. Target names, versions and identifying artifacts are withheld. Each dataset stands on its own; function receipt, static candidates and dynamic validation measure different things.
Target A
Three recorded workflowsThree exploratory sessions with different durations, instructions and worker resources. Direct code/assembly and additional description-only functions are shown separately. The counts describe material returned to the agents, not equivalent completed investigations. [1]
Reported static candidates
Retained static IDs · conditional items included
- Granite Desktop19
- REAopencode3
- IDA MCPopencode1
Finding criteria differ across workflows. These counts do not form a quality ranking or a count of dynamically verified vulnerabilities.
Investigation outcomes · Target A
Follow-up that ruled out a hypothesis
- Candidate inventory
- 19 Granite static IDs form 18 provisional root-cause families.
- Hypothesis refuted
- A follow-up branch refuted one hypothesis.
- Question unresolved
- Another question remained unresolved after human guidance.
Recorded research outcomes; analyst review time was not measured. [1]
Direct code or assembly
Returned function counts
- Granite Desktop657
- REAopencode24
- IDA MCPopencode43
Recorded model usage
Raw agents + engine, USD
- Granite Desktop$14.16
- REAopencode$0.81
- IDA MCPopencode$1.45
Corrected rates. Setup, licensing, infrastructure and analyst review are outside these totals.
Returned evidence and static candidates
Scroll to compare columns| Workflow | Direct functions | Description-only | Combined union | Static IDs |
|---|---|---|---|---|
| Granite Desktop | 657 | 735 | 1,392 | 19 |
| REAopencode | 24 | 0 | 24 | 3 |
| IDA MCPopencode | 43 | 0 | 43 | 1 |
The 19 Granite static IDs form 18 provisional root-cause families and include conditional items. The other workflows report three conditional findings and one principal static finding; one location overlaps a Granite finding. These counts use different finding criteria and do not form a quality ranking. [1]
Recorded resources and corrected model-use equivalents
Scroll to compare columns| Workflow | Elapsed minutes | Code-bearing surfaces | AgentsUSD | EngineUSD | TotalUSD |
|---|---|---|---|---|---|
| Granite Desktop | 156.9 | 21 | $12.86 | $1.30 | $14.16 |
| REAopencode | 55.9 | 6 | $0.81 | $0.00 | $0.81 |
| IDA MCPopencode | 30.4 | 6 | $1.45 | $0.00 | $1.45 |
Granite’s corrected total is $14.16: $12.86 agents plus $1.30 engine. Title generation, cancelled usage and unsuccessful research remain included. A follow-up branch refuted one hypothesis and left another question unresolved after human guidance. Analyst review time was not measured. [1]
Granite used 15 descendant contexts alongside its parent context; peak simultaneous workers are not reported. IDA MCP had a one-worker ceiling. Code-bearing surfaces include support and local-path code, not just entry points or independent products.
Target B
Four recorded workflowsFour recorded workflows, with complete bodies, excerpts, assembly and semantic descriptions counted as separate evidence tiers. The earlier two- and three-workflow reports reuse these same arms; they are included here once. [1]
The REA-labelled workflow is a hybrid: it temporarily used Granite pseudocode extraction. Script-only scans, names and sink flags do not count as returned function bodies. The workflows disagree on some candidates; these counts retain each workflow’s classifications. [1]
Reported static OOB candidates
Out-of-bounds candidates · not dynamic confirmations
- Granite Desktop16
- REA-labelledHybridopencode2
- Granite MCPopencode13
- IDA MCPopencode5
Each bar retains its workflow’s reported classifications. Candidate verdicts conflict across workflows and have not been adjudicated.
Investigation outcomes · Target B
16 Desktop candidates. 13 through MCP.
- Granite Desktop
- 16 reported static out-of-bounds candidates.
- Granite MCP
- 13 reported static out-of-bounds candidates.
- Cross-workflow review
- The workflows disagree on some candidates. The counts retain each workflow’s classifications.
The public data does not assign the separately verified RCE to a target or workflow row. [1] [2]
Complete function bodies
Returned function counts
- Granite Desktop755
- REA-labelledHybridopencode26
- Granite MCPopencode337
- IDA MCPopencode600
Recorded model usage
Raw agents + engine, USD
- Granite Desktop$5.56
- REA-labelledHybridopencode$3.82
- Granite MCPopencode$5.20
- IDA MCPopencode$9.63
Raw usage shown. IDA MCP’s analytically adjusted total is $7.32; the deduction is detailed below. Setup and analyst review are outside these totals.
Investigation depth and reported static candidates
Scroll to compare columns| Workflow | Complete bodies | Any code / assembly | Bodies or descriptions | Static OOB candidates |
|---|---|---|---|---|
| Granite Desktop | 755 | 766 | 938 | 16 |
| REA-labelledHybridopencode | 26 | 82 | 26 | 2 |
| Granite MCPopencode | 337 | 339 | 662 | 13 |
| IDA MCPopencode | 600 | 723 | 600 | 5 |
Raw and analytically adjusted model-use equivalents, USD
Scroll to compare columns| Workflow | Agents | Engine | Raw total | Adjusted total |
|---|---|---|---|---|
| Granite Desktop | $4.84 | $0.72 | $5.56 | $5.56 |
| REA-labelledHybridopencode | $3.82 | $0.00 | $3.82 | $3.82 |
| Granite MCPopencode | $4.56 | $0.63 | $5.20 | $5.20 |
| IDA MCPopencode | $9.63 | $0.00 | $9.63 | $7.32 |
For IDA MCP, the adjusted view deducts $2.31277386 for 209 model turns whose tool calls returned only a capacity error. A mixed turn with a successful write remains charged. The raw total is $9.62812536; the adjusted total is $7.31535150. This is an analytical deduction from the same run, not a rerun or measured savings from a different configuration. Other rows are unchanged. [1]
Detailed counting tiers and function sizes
Additional evidence tiers
Scroll to compare columns| Workflow | Excerpt-only | Assembly-only | Describedincl. overlap | Any code or descriptions |
|---|---|---|---|---|
| Granite Desktop | 4 | 7 | 555 | 949 |
| REA-labelledHybridopencode | 51 | 5 | 0 | 82 |
| Granite MCPopencode | 1 | 1 | 555 | 663 |
| IDA MCPopencode | 120 | 3 | 0 | 723 |
Complete-body content and surface counts
Scroll to compare columns| Workflow | Body characters | Median characters | Bodies under 200 characters | Surfaces with bodies |
|---|---|---|---|---|
| Granite Desktop | 2,921,431 | 1,986 | 91 | 39 |
| REA-labelledHybridopencode | 82,737 | 1,366 | 0 | 10 |
| Granite MCPopencode | 1,542,398 | 3,035 | 20 | 39 |
| IDA MCPopencode | 1,609,292 | 1,558 | 41 | 39 |
Union columns deduplicate functions across their stated tiers. A description is contextual information, not another complete body. Body length and short-function counts provide sensitivity views, not correctness scores. The data download preserves exact numeric values. [1]
One RCE verified. Harness required.
Verification scope
Our team dynamically verified one RCE using a required test harness. Verification followed the recorded static benchmark sessions and applies to that harness setup. [2]
The tables retain their static candidate counts; the dynamically verified result is recorded separately.
Target names and issue-specific reproduction details are excluded from this public benchmark dataset.
The single verified result cannot establish a confirmation rate for the candidate inventory, recall or a false-positive rate. Target identities remain withheld for both datasets.
Run conditions and accounting
All tables use corrected supplied token rates. Agent uncached input / cached input / output rates are $0.50 / $0.08 / $2.00 per million tokens; engine rates are $0.10 / $0.01 / $0.50. Values are usage equivalents, not verified invoices, customer prices or full task costs. Infrastructure, subscriptions, licensing, setup and analyst review are outside these denominators. [1]
Cost per function is an auxiliary process measure. A function receipt does not establish complete review, and different workflows pursued different depths of investigation. Function counts, candidate counts and descriptions cannot be treated as interchangeable units of useful work.
Prompts, stopping rules, worker resources, setup delays and target coverage differ. The recordings do not support a controlled causal claim, pooled detection rate or confidence interval based on individual functions. A controlled comparison requires matched target-task pairs, each system’s intended orchestration, and blinded assessment of accepted outcomes and reviewer effort.
Accounting audits check the packaged usage and credited ledgers, not vulnerability validity or comprehension. Target B’s source accounting audit passes 59 checks. Retained session exports omit streaming deltas and repeated request fields; the public download contains aggregate measurements, not original traces. [1]
Data & sources
This page consolidates both current benchmark datasets and all seven recorded workflow arms from our benchmark repository. Superseded pricing and provisional attribution are replaced by the current accounting; adjusted views are shown alongside raw values. Individual target identifiers, binary names and finding artifacts are withheld.
- Anonymous benchmark data, measurement snapshot 17 November 2026. Includes both datasets, every workflow row, counting tiers, costs, adjustment and rate assumptions. Source references A and B correspond to retained internal reports and accounting audits.
- Binary Scryer validation update, 18 November 2026: our team dynamically verified one RCE using a required test harness. The public dataset records this outcome and its harness requirement; issue-specific reproduction records are not included.
Glossary
- Static analysis
- Examining code without executing the behavior under study.
- Dynamic verification
- Checking a claimed behavior by executing it under stated conditions.
- Test harness
- Supporting code or setup used to exercise and observe behavior under test.
- RCE
- Remote code execution. The verified result requires a test harness.
- OOB
- Out-of-bounds memory access. A static OOB candidate is not automatically a dynamically verified vulnerability.
- Root-cause family
- Reports grouped because they describe the same underlying defect or mechanism.
- Refutation
- Evidence contradicting a proposed explanation.
- Observational comparison
- Recorded runs compared without controlling all relevant conditions.
- Recall and false-positive rate
- Measures requiring independently established positive and negative reference cases, unavailable in these datasets.