control-coverage

Control-first coverage and blind-spot analysis over an evidence corpus.

The rest of the Audit Labs toolchain is evidence-first: audit-tools collects raw signals, audit-report maps each finding onto the controls it touches, and evidence-seal proves the package is authentic. That answers "what did I collect, and what does it map to?" — but it can never tell you what you are not looking at, because it has no list of everything a framework requires.

control-coverage supplies that missing list — the denominator. It starts from the complete catalog of a framework's controls and scores your evidence against it, so it can report two numbers nothing else in the pipeline can:

• Coverage % — of everything the framework requires, how much the evidence corpus addresses at all.
• Blind spots — the in-scope controls that no finding touches. These are the gaps an auditor finds for you if you don't find them first.

It also produces a Statement of Applicability — the ISO 27001 artifact that lists every Annex A control, whether it applies, and why — derived from your evidence instead of hand-maintained.

Like every Audit Labs tool, this produces evidence, not a verdict. An unaddressed control is a gap in evidence, which may reflect a real gap in controls or simply a signal not yet collected. The final judgment belongs to the organization and its auditor.

Install

pip install control-coverage

Pure standard library plus PyYAML — no other dependencies. To hack on it from a clone instead, see Development.

Usage

The input is one or more JSON reports from audit-report (its --format json output). A corpus is typically one report per platform and date — AWS, GitHub, GitLab — which control-coverage folds into a single per-framework picture.

# Coverage across every framework the corpus cites, Markdown to stdout
control-coverage aws.json github.json

# Just the blind spots — the controls nothing evidences yet
control-coverage aws.json github.json --framework SOC2 --blind-spots

# A whole directory of reports, all formats into ./out/
control-coverage ./reports/ --format md,html,json,soa --out out/

# Gate CI: exit non-zero if any framework's coverage is under 60%
control-coverage ./reports/ --fail-under 60

Trend — how coverage moved

Point --baseline at an earlier corpus (a file or a directory) to see what changed: controls that improved, regressed, and — the two that move the coverage number — were gained (a blind spot became addressed) or lost (an addressed control became a blind spot).

control-coverage ./reports/2026-02/ --baseline ./reports/2026-01/ --framework SOC2

# Gate CI: fail the build if any control regressed or lost coverage
control-coverage ./reports/2026-02/ --baseline ./reports/2026-01/ --fail-on-regression

Trend mode outputs Markdown, HTML, or JSON (--format md,html,json).

Crosswalk — evidence leverage and the minimal set

One check is rarely worth one control: enforced 2FA is evidence for SOC 2 CC6.1, ISO A.5.17, and NIST IA-2 at once. --crosswalk shows that leverage per check and computes the minimal evidence set — the fewest checks that still touch every addressed control, which is what you want when scoping a walkthrough or a sample.

control-coverage ./reports/ --framework SOC2,ISO,NIST --crosswalk

Crosswalk mode outputs Markdown, HTML, or JSON (--format md,html,json).

Scope and the Statement of Applicability

Not every control applies to every organization. A scope file records which controls are excluded and — required, never optional — why:

# soa.yaml
subject: Acme Production
frameworks: [SOC2, ISO]
exclusions:
  - control: ISO:A.7.1
    reason: "Fully cloud-hosted; no physical premises are in scope for the ISMS."
  - control: ISO:A.5.7
    reason: "No formal threat-intelligence program; risk accepted by the CISO for 2026."
owners:
  SOC2:CC6.1: platform-team

# Coverage over in-scope controls, plus a ready-to-file SoA
control-coverage ./reports/ --scope soa.yaml --format md,soa --out out/

Excluded controls are recorded with their justification rather than counted as gaps. An exclusion with no reason is rejected — an unjustified exclusion is the single most common SoA audit finding.

Assurance states

Every in-scope control lands in exactly one state:

| State | Meaning | | --- | --- | | supported | At least one mapped finding passes, and none fail. | | failing | At least one mapped finding fails. The worst observation wins. | | asserted | Findings map here, but their data was absent — evidence attempted, not obtained. | | unaddressed | No finding maps here at all. The blind spot. | | out of scope | Excluded by the scope file, with a recorded justification. |

coverage % is the share of in-scope controls in any of the first three states; assured % is the share that are supported.

Bundled catalogs

| Framework | Code | Catalog | | --- | --- | --- | | SOC 2 (Trust Services Criteria) | SOC2 | Complete — all five categories (Common Criteria + Availability, Confidentiality, Processing Integrity, Privacy), 61 controls | | ISO/IEC 27001:2022 Annex A | ISO | Complete — all 93 controls | | NIST SP 800-53 Rev. 5 | NIST | Moderate baseline — 177 base controls across 18 families |

Most SOC 2 reports scope only some categories (Security is near-universal; Privacy and Processing Integrity often are not). Use exclude_families in the scope file to drop a whole category — or an ISO theme, or a NIST family — from the denominator in one line:

exclude_families:
  - {framework: SOC2, family: Privacy, reason: "Privacy category not in the SOC 2 audit scope."}
  - {framework: SOC2, family: Processing Integrity, reason: "PI category not in the SOC 2 audit scope."}

Control codes are written FRAMEWORK:ID (SOC2:CC6.1, ISO:A.5.17), matching the codes audit-report rulesets already cite. A partial catalog is reported honestly as coverage of the shipped subset, never as the whole standard.

If the corpus cites a code whose framework is loaded but the catalog does not define it — a typo or a renamed control — it is surfaced as an unmatched control code rather than silently ignored.

How it fits the pipeline

audit-tools ──► CSV package ──► evidence-seal (seal + verify)
                     │
                     ▼
               audit-report ──► per-package report (--format json)
                                        │
                                        ▼  one or more reports = a corpus
                               control-coverage ──► coverage %, blind spots, SoA,
                                                    trend over time, evidence crosswalk

Stability

control-coverage is stable as of v1.0.0 and follows semantic versioning. It reads audit-report's v1 JSON contract; the bundled catalogs, the coverage / Statement of Applicability / JSON output schemas, and the --fail-under gate are committed within the 1.x line.

Development

pip install -e ".[dev]"
pytest
ruff check .

License

GPL-3.0-or-later. See LICENSE.