Scorecard schema

A scorecard is the structured output of anc audit <binary> --output json. The site reads scorecards under scorecards/ and renders them at /scorecards and at each tool's /score/<name> page. This page documents every field that appears in a scorecard, what it means, and where it comes from. It is intentionally exhaustive: anything in the JSON should be explainable from this page.

Filename

Scorecards are stored on disk as:

scorecards/<name>-v<version>.json

Where <name> matches the registry's name field (URL slug) and <version> is the SemVer string captured at scoring time. The filename's <version> segment is the canonical version anchor: the site reads it directly off disk and displays it as the scored version on every per-tool page. The scorecard's tool.version field (added in schema 0.4) is informational; when both are present and disagree, the build aborts with an integrity error. The filename never lies.

Top-level fields

{
  "schema_version": "0.6",
  "spec_version": "...",
  "tool":   { "name": "...", "binary": "...", "version": "..." },
  "anc":    { "version": "..." },
  "run":    { "invocation": "...", "started_at": "...", "duration_ms": 0,
              "platform": { "os": "...", "arch": "..." } },
  "target": { "kind": "...", "path": null, "command": "..." },
  "audience": "...",
  "audience_reason": null,
  "audit_profile": null,
  "summary": { ... },
  "coverage_summary": { ... },
  "badge": { ... },
  "results": [ ... ]
}

tool

Self-describing tool identity. Lets a downstream consumer answer "what was scored?" without cross-referencing the registry.

"tool": {
  "name": "rg",
  "binary": "rg",
  "version": "ripgrep 15.1.0"
}

Build-time invariant: when tool.version contains a SemVer-shaped token (X.Y or X.Y.Z), it must equal the filename version. Drift fails the build with a parser-asymmetry error: the regen script's version_extract snippet and the CLI's internal probe are the only two places that derive a version from the binary, and they must agree.

anc

Provenance for the scorecard: which anc build produced it.

"anc": {
  "version": "0.2.1"
}

The per-tool page renders anc.version.

run

Run-context: the literal invocation, when it ran, how long it took, and what platform it ran on.

"run": {
  "invocation": "anc audit --command rg --output json",
  "started_at": "2026-04-30T04:18:53.099683344Z",
  "duration_ms": 53,
  "platform": { "os": "linux", "arch": "x86_64" }
}

Security note (run.invocation): for command-mode runs the invocation is the canonical anc audit --command <name> [--audit-profile <X>] [--output json] shape, which is safe to embed publicly. For project-mode runs (target.kind: "project") the invocation may include a local filesystem path (anc audit ./local/repo); the site falls back to the synthesized form for those runs to avoid leaking machine-local paths into HTML, markdown, and /llms-full.txt. Mirror this gate downstream if you fetch the JSON directly.

target

What the run was scoring: a command, a binary on disk, or a project tree.

"target": {
  "kind": "command",
  "path": null,
  "command": "rg"
}

Security note (target.path): when kind is project, this can carry a local directory path (/home/me/dev/foo). It is not currently rendered on any per-tool page (every leaderboard entry today is command-mode), but downstream consumers reading the JSON should treat it as machine-local.

summary

Counts of how the runner's audits finished, one per evaluated requirement row. Adds up to total.

"summary": {
  "total": 18,
  "pass": 14,
  "warn": 1,
  "fail": 0,
  "opt_out": 1,
  "n_a": 0,
  "skip": 2,
  "error": 0
}

The leaderboard score counts pass, warn, fail, and opt_out in the denominator and credits pass full and warn half; n_a, skip, and error are excluded from both sides, and only behavioral-layer rows are scored. The full formula is on the methodology page.

coverage_summary

Tally of how much of the agentnative spec the run verified. Distinct from summary, which counts the runner's audits. One implemented audit can verify zero, one, or many spec requirements; many spec requirements aren't yet covered by any implemented audit.

"coverage_summary": {
  "must":   { "total": 23, "verified": 9 },
  "should": { "total": 16, "verified": 0 },
  "may":    { "total": 7,  "verified": 0 }
}

If coverage_summary.must.verified is below summary.pass, that's expected because a single passing audit can map to multiple MUSTs. If should.verified and may.verified are zero across the board, that's also expected: those tiers are aspirational and will fill in as the runner grows audits mapped to them.

badge

The leaderboard score, its eligibility verdict, and the ready-to-paste embed snippet. anc derives these from results so a consumer can read the score without recomputing it.

"badge": {
  "eligible": true,
  "score_pct": 84,
  "embed_markdown": "[![agent-native](https://anc.dev/badge/rg.svg)](https://anc.dev/score/rg)",
  "scorecard_url": "https://anc.dev/score/rg",
  "badge_url": "https://anc.dev/badge/rg.svg",
  "convention_url": "https://anc.dev/badge"
}

The site reads badge.score_pct and badge.eligible straight from the scorecard rather than recomputing them, so the number a third party reads matches the number anc produced. See the badge convention for the embed contract and the cohort-band colors.

results

Array of one object per evaluated requirement row. Order is stable across runs of the same anc version. One probe can back several requirement rows (for example, the --version probe answers both a MUST and a SHOULD); each row is scored independently and carries its own tier.

{
  "id": "p3-must-help",
  "label": "Help flag produces useful output",
  "group": "P3",
  "layer": "behavioral",
  "tier": "must",
  "status": "pass",
  "evidence": null,
  "confidence": "high"
}

status semantics in detail

• pass: Requirement met. Full credit.
• warn: Requirement partially satisfied (a missed SHOULD or MAY). Half credit.
• fail: A MUST-tier requirement was not satisfied. No credit, counts in the denominator.
• opt_out: The tool could implement the requirement but deliberately does not. No credit, counts in the denominator.
• n_a: The requirement does not apply: a conditional antecedent is absent, or an audit profile scopes it out. Excluded from the score.
• skip: The probe could not measure the property: a linter limitation, not a tool defect (for example, a JSON-output audit that detected --output but could not validate the payload through safe probes). evidence says why.
• error: Audit tried to run and crashed before producing a verdict. Treated as no-signal, not a defect.

What is not in the scorecard (yet)

The site is transparent about gaps that future schema bumps may fill. Schema 0.4 closed the tool-identity / generated-at gap (see tool, anc, run, target above). Still outstanding today:

• Per-audit timing. run.duration_ms is the wall-clock total for the run, not per-audit. Individual audit timings are observable from the runner's stdout but not captured in the JSON.
• Editorial fields inside the scorecard. Language, creator, description, install, and repo/url remain in the registry. Migrating them into the scorecard would let the registry shrink to a name list; deferred to a future schema bump.

When the anc CLI grows fields for any of the above, the schema page above will be updated and schema_version will bump.

Why is audience null for some tools?

When a tool is scored with audit_profile: human-tui, anc suppresses one or more of the four signal audits the audience classifier consumes. With incomplete inputs, the classifier emits audience: null and sets audience_reason: "suppressed" rather than guess. This is correct, intentional, and shared across every TUI on the leaderboard. The methodology page covers the full classifier definition.