Audit Labs / walkthrough

One finding,
the whole pipeline.

A worked end-to-end example. We collect evidence from a fictional company’s cloud, seal it, map it to controls, and measure coverage — and follow a single risky account the whole way down, so you can see exactly what each tool hands to the next.

# 1 · COLLECT — pull evidence from live systems
$ python audit_tui.py output/aws_audit_northwind_2026-07-29/
# 2 · SEAL — anchor chain of custody
$ evidence-seal seal ./output/aws_audit_northwind_2026-07-29
  sealed 6 files  root 1265ef6f…
# 3 · REPORT — map findings to controls
$ audit-report ./output/aws_audit_northwind_2026-07-29 --format json
  12 checks · 3 exceptions reports/aws.json
# 4 · MEASURE — score the corpus vs the framework
$ control-coverage reports/ --framework SOC2
  SOC 2 · 63% covered · 4 blind spots
The scenario

Northwind, a SaaS company, mid-SOC 2 fieldwork.

Northwind runs on AWS, GitHub, and Postgres. You’re gathering evidence for a SOC 2 Type II examination covering Q3 2026. Rather than trace every file, we’ll follow one account that turns out to matter — and watch it change shape at each handoff.

Follow this row → j.hale — an IAM user with console access and no MFA
Step 1 · Collect  —  audit-tools

Raw evidence, straight from the systems.

audit-tools runs read-only collectors against each platform and writes plain CSVs into one dated package directory — here, aws_audit_northwind_2026-07-29/. No judgments yet: just the facts as the systems report them, one file per procedure.

In iam_users.csv, most users are fine. One row isn’t: j.hale can log in to the console but has no second factor. That’s our thread.

# Interactive: pick platform, checks, credentials
python audit_tui.py

# → writes a package of CSVs
output/aws_audit_northwind_2026-07-29/
  iam_users.csv  password_policy.csv  account_security.csv
  open_security_groups.csv  config_recorders.csv  cloudtrail.csv
audit-tools on GitHub →
iam_users.csv6-file package
userconsole_passwordmfa_enabledoldest_key_age_days
a.okaforTrueTrue62
j.haleTrueFalse404
svc-deployFalse91
r.mendezTrueTrue15
Step 2 · Seal  —  evidence-seal  (optional)

Freeze the package before anyone touches it.

The moment collection finishes, seal the directory. evidence-seal hashes every file into a single Merkle root. Months later, verify proves the package is byte-for-byte what you collected — and names anything that moved.

If someone later edits j.hale’s row to look compliant, the root no longer matches and verify fails loudly. Re-seal each collection round with --prev to chain a Type II custody history.

# Seal, then (much later) verify
evidence-seal seal ./output/aws_audit_northwind_2026-07-29
evidence-seal verify ./output/aws_audit_northwind_2026-07-29
evidence-seal details →
verify
$ evidence-seal seal ./output/aws_audit_northwind_2026-07-29
  sealed 6 files
  root 1265ef6f…  id d096c7ed…
──────────────────────────────
$ evidence-seal verify ./…northwind_2026-07-29
  ✓ intact — 6 files match the seal
# someone edited j.hale’s row?
$ evidence-seal verify ./…northwind_2026-07-29
  ✗ TAMPERED — MODIFIED iam_users.csv
Step 3 · Report  —  audit-report

Turn rows into findings, mapped to controls.

audit-report reads the CSV package and applies a declarative ruleset. A rule names a table, a check, and the controls the signal is evidence for. The iam-console-no-mfa rule fails j.hale and maps it to SOC2:CC6.1, NIST:IA-2, and ISO:A.5.17.

JSON feeds the next stage; the same run also emits an auditor-ready HTML report. Add --baseline to diff against last quarter and flag drift.

# Machine-readable JSON for the next stage
audit-report ./output/aws_audit_northwind_2026-07-29 \
  --format json,html --out reports/

12 checks · 3 exceptions → reports/aws.json
audit-report details →
reports/aws.jsonone finding
"rule": "iam-console-no-mfa", "status": "fail", "severity": "high", "controls": ["SOC2:CC6.1", "NIST:IA-2", "ISO:A.5.17"], "evidence": { "table": "iam_users", "rows": [ { "user": "j.hale", "console_password": true, "mfa_enabled": false } ] }
Step 4 · Measure  —  control-coverage

Score the corpus against the whole framework.

Every earlier tool is evidence-first — it can only speak to what you collected. control-coverage supplies the missing denominator: the complete catalog of a framework’s controls. Feed it the report JSON(s) and it reports coverage %, the blind spots nothing touches, and a Statement of Applicability.

The payoff for our thread: CC6.1 is now addressed — even as an exception, the control is no longer a blind spot, because step 3 produced a finding mapped to it. The blind spots are the controls no collector reached at all.

# Fold all reports into one framework picture
control-coverage reports/ --framework SOC2 \
  --format md,html,soa --out out/

# Gate CI under a coverage floor
control-coverage reports/ --fail-under 60
control-coverage details →
SOC 2 — coverageout/coverage.md
Trust Services Criteria touched63%
Addressed by evidence
CC6.1 CC6.2 CC6.6 CC7.1 CC7.2 CC8.1
Blind spots — no evidence collected
CC1.4  Background checks / HR onboarding
CC2.3  Communicating objectives to external parties
CC9.2  Vendor & third-party risk management
A1.2   Environmental protections
Why it composes

Four stages, three stable contracts.

Each tool cares only about the shape of the artifact it’s handed. That’s why you can run one step or the whole chain — and swap the collector without the rest noticing.

CSV package
audit-toolsevidence-seal & audit-report. A dated directory of plain CSVs — the one artifact everything downstream reads.
sealed package
evidence-seal wraps the package with a Merkle manifest — optional, and transparent to the stages after it.
findings JSON
audit-reportcontrol-coverage. One JSON per platform/date; a folder of them is a corpus.
coverage + SoA
control-coverage emits the numbers nothing upstream can: coverage %, blind spots, Statement of Applicability.

Swap the collector, free of charge. gh-attest is the continuous variant of step 1 — a read-only GitHub App that captures branch protection and access over time and exports the same CSV-package shape. It drops straight into steps 2–4 unchanged.

Built to produce evidence for
SOX SOC 1 / 2 ISO 27001 NIST 800-53 ITGC