audit-labs/audit-labs.dev
The website for Audit Labs.
clone: git clone https://gitbay.org/audit-labs/audit-labs.dev.git
main: walkthrough/index.html · raw
1<!DOCTYPE html>
2<html lang="en">
3<head>
4 <meta charset="UTF-8">
5 <meta name="viewport" content="width=device-width, initial-scale=1.0">
6 <title>End-to-end walkthrough | Audit Labs</title>
7 <meta name="description" content="Watch one finding travel the whole Audit Labs pipeline — collect it with audit-tools, seal it with evidence-seal, map it to controls with audit-report, and measure coverage with control-coverage.">
8 <link rel="preconnect" href="https://fonts.googleapis.com">
9 <link href="https://fonts.googleapis.com/css2?family=IBM+Plex+Mono:wght@400;500;700&family=DM+Serif+Display:ital@0;1&family=DM+Sans:wght@400;500;600&display=swap" rel="stylesheet">
10 <link rel="stylesheet" href="../assets/site.css">
11 <style>
12 /* ---- walkthrough-specific layout (scoped to this page) ---- */
13 .stage { display: grid; grid-template-columns: 1fr 1fr; gap: 2.6rem; margin-top: 2.4rem; align-items: start; }
14 .stage > * { min-width: 0; }
15 @media (max-width: 860px) { .stage { grid-template-columns: 1fr; gap: 1.7rem; } }
16
17 .stage-num { font-family: 'IBM Plex Mono', monospace; font-size: 0.68rem; letter-spacing: 0.14em; text-transform: uppercase; color: var(--amber); margin-bottom: 0.9rem; display: flex; align-items: center; gap: 0.6rem; }
18 .stage-num b { font-weight: 700; color: var(--black); background: #fef3c7; border-radius: 3px; padding: 0.05rem 0.45rem; }
19 .stage-lede { color: var(--gray-3); font-size: 0.98rem; line-height: 1.7; margin-top: 0.7rem; }
20 .stage-lede .mono { color: var(--gray-1); font-family: 'IBM Plex Mono', monospace; font-size: 0.86em; }
21 .stage-lede .ctrl { color: var(--amber); font-family: 'IBM Plex Mono', monospace; font-size: 0.84em; }
22
23 .stage pre { background: #111; color: #e8e0d0; border-radius: 8px; padding: 1.1rem 1.3rem; font-family: 'IBM Plex Mono', monospace; font-size: 0.76rem; line-height: 1.85; overflow-x: auto; margin-top: 1.3rem; }
24 .stage pre .c { color: #555; } .stage pre .p { color: #5a9060; } .stage pre .o { color: #7a9f7a; } .stage pre .h { color: #8a8a8a; }
25
26 /* artifact panel — looks like a small file/output viewer */
27 .panel { border: 1px solid var(--border); border-radius: 8px; overflow: hidden; background: #fff; }
28 .panel-bar { background: var(--off-white); border-bottom: 1px solid var(--border); padding: 0.5rem 0.9rem; font-family: 'IBM Plex Mono', monospace; font-size: 0.68rem; color: var(--gray-3); display: flex; align-items: center; gap: 0.55rem; }
29 .panel-bar .fname { color: var(--gray-1); font-weight: 700; }
30 .panel-bar .tag { margin-left: auto; letter-spacing: 0.08em; text-transform: uppercase; color: var(--gray-4); }
31 .panel-body { overflow-x: auto; }
32
33 table.evi { border-collapse: collapse; font-family: 'IBM Plex Mono', monospace; font-size: 0.7rem; width: 100%; white-space: nowrap; }
34 table.evi th { text-align: left; color: var(--gray-4); font-weight: 500; padding: 0.5rem 0.85rem; border-bottom: 1px solid var(--border); }
35 table.evi td { padding: 0.45rem 0.85rem; border-bottom: 1px solid var(--border); color: var(--gray-1); }
36 table.evi tr:last-child td { border-bottom: none; }
37 tr.hl td { background: #fef3c7; color: var(--black); }
38 .bad { color: var(--red); font-weight: 700; }
39 .ok { color: var(--green); }
40
41 /* json artifact */
42 .json { font-family: 'IBM Plex Mono', monospace; font-size: 0.73rem; line-height: 1.75; padding: 1rem 1.15rem; white-space: pre; color: #4a4a4a; }
43 .json .jk { color: var(--gray-1); } .json .js { color: var(--green); } .json .jn { color: var(--amber); }
44 .json .jc { color: var(--amber); font-weight: 700; }
45 .json-hl { background: #fef3c7; display: block; border-radius: 3px; }
46
47 /* coverage meter */
48 .cov { padding: 1.2rem 1.15rem; }
49 .cov-row { display: flex; align-items: baseline; justify-content: space-between; font-family: 'IBM Plex Mono', monospace; font-size: 0.74rem; color: var(--gray-1); margin-bottom: 0.5rem; }
50 .cov-row b { font-size: 1.05rem; color: var(--black); }
51 .meter { height: 8px; background: var(--off-white); border: 1px solid var(--border); border-radius: 999px; overflow: hidden; margin-bottom: 1.2rem; }
52 .meter > span { display: block; height: 100%; width: 63%; background: linear-gradient(90deg, var(--amber), var(--amber-light)); }
53 .cov h5 { font-family: 'IBM Plex Mono', monospace; font-size: 0.64rem; letter-spacing: 0.1em; text-transform: uppercase; color: var(--gray-4); margin: 0.4rem 0 0.55rem; }
54 .chip-row { display: flex; flex-wrap: wrap; gap: 0.35rem; margin-bottom: 1rem; }
55 .chip { font-family: 'IBM Plex Mono', monospace; font-size: 0.68rem; border-radius: 4px; padding: 0.16rem 0.5rem; border: 1px solid var(--border); color: var(--gray-3); background: #fff; }
56 .chip.on { color: var(--green); border-color: rgba(21,128,61,0.35); background: rgba(21,128,61,0.06); }
57 .chip.here { color: var(--amber); border-color: rgba(217,119,6,0.4); background: #fef3c7; font-weight: 700; }
58 .blind { font-family: 'IBM Plex Mono', monospace; font-size: 0.7rem; color: var(--gray-3); line-height: 1.9; }
59 .blind b { color: var(--red); }
60
61 /* through-line legend */
62 .thread { display: flex; align-items: center; gap: 0.8rem; flex-wrap: wrap; margin-top: 2rem; background: #fff; border: 1px solid var(--border); border-left: 3px solid var(--amber); border-radius: 6px; padding: 0.85rem 1.1rem; }
63 .thread .lab { font-family: 'IBM Plex Mono', monospace; font-size: 0.64rem; letter-spacing: 0.1em; text-transform: uppercase; color: var(--amber); }
64 .thread .val { font-family: 'IBM Plex Mono', monospace; font-size: 0.78rem; color: var(--gray-1); }
65 .thread .val b { color: var(--black); }
66
67 .stage-link { display: inline-block; margin-top: 1.2rem; font-family: 'IBM Plex Mono', monospace; font-size: 0.74rem; color: var(--gray-3); text-decoration: none; border-bottom: 1px solid var(--gray-5); padding-bottom: 1px; transition: color 0.15s; }
68 .stage-link:hover { color: var(--amber); }
69
70 .contract-list { margin-top: 2.2rem; }
71 section.alt { background: var(--warm-white); }
72 </style>
73</head>
74<body>
75
76<nav>
77 <div class="nav-inner">
78 <a href="/" class="logo">AUDIT<span>LABS</span></a>
79 <ul class="nav-links">
80 <li><a href="https://github.com/audit-labs" class="btn-github">GitHub →</a></li>
81 </ul>
82 </div>
83</nav>
84
85<header class="hero split">
86 <div class="hero-inner">
87 <div>
88 <div class="crumb"><a href="/">Audit Labs</a> / walkthrough</div>
89 <h1>One finding,<br><em>the whole pipeline.</em></h1>
90 <p class="hero-desc">
91 A worked end-to-end example. We collect evidence from a fictional company’s
92 cloud, seal it, map it to controls, and measure coverage — and follow a
93 <em>single</em> risky account the whole way down, so you can see exactly what each
94 tool hands to the next.
95 </p>
96 <div class="hero-actions">
97 <a href="#step1" class="btn-primary">Start at step 1</a>
98 <a href="/#repos" class="btn-secondary">The tools</a>
99 </div>
100 </div>
101 <div class="terminal">
102 <div class="terminal-bar"><div class="dot dot-red"></div><div class="dot dot-yellow"></div><div class="dot dot-green"></div></div>
103 <div class="terminal-body">
104 <span class="t-comment"># 1 · COLLECT — pull evidence from live systems</span><br>
105 <span class="t-prompt">$</span> <span class="t-cmd">python audit_tui.py</span> <span class="t-dim">→</span> <span class="t-path">output/aws_audit_northwind_2026-07-29/</span><br>
106 <span class="t-comment"># 2 · SEAL — anchor chain of custody</span><br>
107 <span class="t-prompt">$</span> <span class="t-cmd">evidence-seal seal ./output/aws_audit_northwind_2026-07-29</span><br>
108 <span class="t-out"> sealed 6 files root </span><span class="t-hash">1265ef6f…</span><br>
109 <span class="t-comment"># 3 · REPORT — map findings to controls</span><br>
110 <span class="t-prompt">$</span> <span class="t-cmd">audit-report ./output/aws_audit_northwind_2026-07-29 --format json</span><br>
111 <span class="t-out"> 12 checks · 3 exceptions</span> <span class="t-dim">→</span> <span class="t-path">reports/aws.json</span><br>
112 <span class="t-comment"># 4 · MEASURE — score the corpus vs the framework</span><br>
113 <span class="t-prompt">$</span> <span class="t-cmd">control-coverage reports/ --framework SOC2</span><br>
114 <span class="t-out"> SOC 2 · 63% covered · 4 blind spots</span>
115 </div>
116 </div>
117 </div>
118</header>
119
120<!-- SCENARIO -->
121<section>
122 <div class="section-inner">
123 <div class="eyebrow">The scenario</div>
124 <h2>Northwind, a SaaS company, mid-SOC 2 fieldwork.</h2>
125 <p class="lede">
126 Northwind runs on AWS, GitHub, and Postgres. You’re gathering evidence for a SOC 2
127 Type II examination covering Q3 2026. Rather than trace every file, we’ll follow
128 one account that turns out to matter — and watch it change shape at each handoff.
129 </p>
130 <div class="thread">
131 <span class="lab">Follow this row →</span>
132 <span class="val"><b>j.hale</b> — an IAM user with <b>console access</b> and <b>no MFA</b></span>
133 </div>
134 </div>
135</section>
136
137<!-- STEP 1 · COLLECT -->
138<section id="step1" class="alt">
139 <div class="section-inner">
140 <div class="stage-num"><b>Step 1</b> · Collect — audit-tools</div>
141 <h2>Raw evidence, straight from the systems.</h2>
142 <div class="stage">
143 <div>
144 <p class="stage-lede">
145 <span class="mono">audit-tools</span> runs read-only collectors against each platform and
146 writes plain CSVs into one dated package directory — here,
147 <span class="mono">aws_audit_northwind_2026-07-29/</span>. No judgments yet: just the
148 facts as the systems report them, one file per procedure.
149 </p>
150 <p class="stage-lede">
151 In <span class="mono">iam_users.csv</span>, most users are fine. One row isn’t:
152 <span class="mono">j.hale</span> can log in to the console but has no second factor. That’s
153 our thread.
154 </p>
155 <pre><span class="c"># Interactive: pick platform, checks, credentials</span>
156python audit_tui.py
157
158<span class="c"># → writes a package of CSVs</span>
159output/aws_audit_northwind_2026-07-29/
160 <span class="o">iam_users.csv password_policy.csv account_security.csv</span>
161 <span class="o">open_security_groups.csv config_recorders.csv cloudtrail.csv</span></pre>
162 <a href="https://github.com/audit-labs/audit-tools" class="stage-link">audit-tools on GitHub →</a>
163 </div>
164 <div class="panel">
165 <div class="panel-bar"><span class="fname">iam_users.csv</span><span class="tag">6-file package</span></div>
166 <div class="panel-body">
167 <table class="evi">
168 <tr><th>user</th><th>console_password</th><th>mfa_enabled</th><th>oldest_key_age_days</th></tr>
169 <tr><td>a.okafor</td><td>True</td><td class="ok">True</td><td>62</td></tr>
170 <tr class="hl"><td>j.hale</td><td>True</td><td class="bad">False</td><td>404</td></tr>
171 <tr><td>svc-deploy</td><td>False</td><td>—</td><td>91</td></tr>
172 <tr><td>r.mendez</td><td>True</td><td class="ok">True</td><td>15</td></tr>
173 </table>
174 </div>
175 </div>
176 </div>
177 </div>
178</section>
179
180<!-- STEP 2 · SEAL -->
181<section>
182 <div class="section-inner">
183 <div class="stage-num"><b>Step 2</b> · Seal — evidence-seal <span style="color:var(--gray-4);text-transform:none;letter-spacing:0">(optional)</span></div>
184 <h2>Freeze the package before anyone touches it.</h2>
185 <div class="stage">
186 <div>
187 <p class="stage-lede">
188 The moment collection finishes, seal the directory. <span class="mono">evidence-seal</span>
189 hashes every file into a single Merkle <span class="mono">root</span>. Months later,
190 <span class="mono">verify</span> proves the package is byte-for-byte what you collected —
191 and names anything that moved.
192 </p>
193 <p class="stage-lede">
194 If someone later edits <span class="mono">j.hale</span>’s row to look compliant, the
195 root no longer matches and <span class="mono">verify</span> fails loudly. Re-seal each
196 collection round with <span class="mono">--prev</span> to chain a Type II custody history.
197 </p>
198 <pre><span class="c"># Seal, then (much later) verify</span>
199evidence-seal seal ./output/aws_audit_northwind_2026-07-29
200evidence-seal verify ./output/aws_audit_northwind_2026-07-29</pre>
201 <a href="/evidence-seal/" class="stage-link">evidence-seal details →</a>
202 </div>
203 <div class="panel">
204 <div class="panel-bar"><div class="dot dot-red"></div><div class="dot dot-yellow"></div><div class="dot dot-green"></div><span class="fname" style="margin-left:.3rem">verify</span></div>
205 <div class="terminal-body" style="background:#111;">
206 <span class="t-prompt">$</span> <span class="t-cmd">evidence-seal seal ./output/aws_audit_northwind_2026-07-29</span><br>
207 <span class="t-out"> sealed 6 files</span><br>
208 <span class="t-out"> root </span><span class="t-hash">1265ef6f…</span><span class="t-out"> id </span><span class="t-hash">d096c7ed…</span><br>
209 <span class="t-dim">──────────────────────────────</span><br>
210 <span class="t-prompt">$</span> <span class="t-cmd">evidence-seal verify ./…northwind_2026-07-29</span><br>
211 <span class="t-out"> ✓ intact — 6 files match the seal</span><br>
212 <span class="t-comment"># someone edited j.hale’s row?</span><br>
213 <span class="t-prompt">$</span> <span class="t-cmd">evidence-seal verify ./…northwind_2026-07-29</span><br>
214 <span class="t-bad"> ✗ TAMPERED — MODIFIED iam_users.csv</span>
215 </div>
216 </div>
217 </div>
218 </div>
219</section>
220
221<!-- STEP 3 · REPORT -->
222<section class="alt">
223 <div class="section-inner">
224 <div class="stage-num"><b>Step 3</b> · Report — audit-report</div>
225 <h2>Turn rows into findings, mapped to controls.</h2>
226 <div class="stage">
227 <div>
228 <p class="stage-lede">
229 <span class="mono">audit-report</span> reads the CSV package and applies a declarative
230 ruleset. A rule names a table, a check, and the controls the signal is evidence
231 <em>for</em>. The <span class="mono">iam-console-no-mfa</span> rule fails
232 <span class="mono">j.hale</span> and maps it to <span class="ctrl">SOC2:CC6.1</span>,
233 <span class="ctrl">NIST:IA-2</span>, and <span class="ctrl">ISO:A.5.17</span>.
234 </p>
235 <p class="stage-lede">
236 JSON feeds the next stage; the same run also emits an auditor-ready HTML report.
237 Add <span class="mono">--baseline</span> to diff against last quarter and flag drift.
238 </p>
239 <pre><span class="c"># Machine-readable JSON for the next stage</span>
240audit-report ./output/aws_audit_northwind_2026-07-29 \
241 --format json,html --out reports/
242
243<span class="o">12 checks · 3 exceptions</span> <span class="c">→ reports/aws.json</span></pre>
244 <a href="/audit-report/" class="stage-link">audit-report details →</a>
245 </div>
246 <div class="panel">
247 <div class="panel-bar"><span class="fname">reports/aws.json</span><span class="tag">one finding</span></div>
248 <div class="json"><span class="jk">"rule"</span>: <span class="js">"iam-console-no-mfa"</span>,
249<span class="jk">"status"</span>: <span class="js">"fail"</span>, <span class="jk">"severity"</span>: <span class="js">"high"</span>,
250<span class="jk">"controls"</span>: [<span class="jc">"SOC2:CC6.1"</span>, <span class="jc">"NIST:IA-2"</span>, <span class="jc">"ISO:A.5.17"</span>],
251<span class="jk">"evidence"</span>: {
252 <span class="jk">"table"</span>: <span class="js">"iam_users"</span>,
253 <span class="jk">"rows"</span>: [<span class="json-hl"> { <span class="jk">"user"</span>: <span class="js">"j.hale"</span>,
254 <span class="jk">"console_password"</span>: <span class="jn">true</span>,
255 <span class="jk">"mfa_enabled"</span>: <span class="jn">false</span> }</span> ]
256}</div>
257 </div>
258 </div>
259 </div>
260</section>
261
262<!-- STEP 4 · MEASURE -->
263<section>
264 <div class="section-inner">
265 <div class="stage-num"><b>Step 4</b> · Measure — control-coverage</div>
266 <h2>Score the corpus against the whole framework.</h2>
267 <div class="stage">
268 <div>
269 <p class="stage-lede">
270 Every earlier tool is evidence-first — it can only speak to what you collected.
271 <span class="mono">control-coverage</span> supplies the missing denominator: the
272 <em>complete</em> catalog of a framework’s controls. Feed it the report JSON(s) and it
273 reports coverage %, the blind spots nothing touches, and a Statement of Applicability.
274 </p>
275 <p class="stage-lede">
276 The payoff for our thread: <span class="ctrl">CC6.1</span> is now
277 <b>addressed</b> — even as an <em>exception</em>, the control is no longer a blind spot,
278 because step 3 produced a finding mapped to it. The blind spots are the controls
279 <em>no</em> collector reached at all.
280 </p>
281 <pre><span class="c"># Fold all reports into one framework picture</span>
282control-coverage reports/ --framework SOC2 \
283 --format md,html,soa --out out/
284
285<span class="c"># Gate CI under a coverage floor</span>
286control-coverage reports/ --fail-under 60</pre>
287 <a href="/control-coverage/" class="stage-link">control-coverage details →</a>
288 </div>
289 <div class="panel">
290 <div class="panel-bar"><span class="fname">SOC 2 — coverage</span><span class="tag">out/coverage.md</span></div>
291 <div class="cov">
292 <div class="cov-row"><span>Trust Services Criteria touched</span><b>63%</b></div>
293 <div class="meter"><span></span></div>
294 <h5>Addressed by evidence</h5>
295 <div class="chip-row">
296 <span class="chip here">CC6.1</span>
297 <span class="chip on">CC6.2</span>
298 <span class="chip on">CC6.6</span>
299 <span class="chip on">CC7.1</span>
300 <span class="chip on">CC7.2</span>
301 <span class="chip on">CC8.1</span>
302 </div>
303 <h5>Blind spots — no evidence collected</h5>
304 <div class="blind">
305 <b>CC1.4</b> Background checks / HR onboarding<br>
306 <b>CC2.3</b> Communicating objectives to external parties<br>
307 <b>CC9.2</b> Vendor & third-party risk management<br>
308 <b>A1.2</b> Environmental protections
309 </div>
310 </div>
311 </div>
312 </div>
313 </div>
314</section>
315
316<!-- WHY IT COMPOSES -->
317<section class="cmds">
318 <div class="section-inner">
319 <div class="eyebrow">Why it composes</div>
320 <h2>Four stages, three stable contracts.</h2>
321 <p class="lede">
322 Each tool cares only about the <em>shape</em> of the artifact it’s handed. That’s why you can
323 run one step or the whole chain — and swap the collector without the rest noticing.
324 </p>
325 <div class="cmd-list contract-list">
326 <div class="cmd-row"><div class="cmd-name">CSV package</div><div class="cmd-desc"><span class="mono">audit-tools</span> → <span class="mono">evidence-seal</span> & <span class="mono">audit-report</span>. A dated directory of plain CSVs — the one artifact everything downstream reads.</div></div>
327 <div class="cmd-row"><div class="cmd-name">sealed package</div><div class="cmd-desc"><span class="mono">evidence-seal</span> wraps the package with a Merkle manifest — optional, and transparent to the stages after it.</div></div>
328 <div class="cmd-row"><div class="cmd-name">findings JSON</div><div class="cmd-desc"><span class="mono">audit-report</span> → <span class="mono">control-coverage</span>. One JSON per platform/date; a folder of them is a <em>corpus</em>.</div></div>
329 <div class="cmd-row"><div class="cmd-name">coverage + SoA</div><div class="cmd-desc"><span class="mono">control-coverage</span> emits the numbers nothing upstream can: coverage %, blind spots, Statement of Applicability.</div></div>
330 </div>
331 <p class="lede" style="margin-top:2rem;">
332 <b>Swap the collector, free of charge.</b> <a href="/gh-attest/" style="color:var(--amber);text-decoration:none;">gh-attest</a>
333 is the continuous variant of step 1 — a read-only GitHub App that captures branch
334 protection and access <em>over time</em> and exports the same CSV-package shape. It drops
335 straight into steps 2–4 unchanged.
336 </p>
337 </div>
338</section>
339
340<!-- FRAMEWORKS strip -->
341<section class="frameworks">
342 <div class="frameworks-inner">
343 <span class="frameworks-label">Built to produce evidence for</span>
344 <div class="frameworks-list">
345 <span class="framework-tag">SOX</span>
346 <span class="framework-tag">SOC 1 / 2</span>
347 <span class="framework-tag">ISO 27001</span>
348 <span class="framework-tag">NIST 800-53</span>
349 <span class="framework-tag">ITGC</span>
350 </div>
351 </div>
352</section>
353
354<footer>
355 <div class="footer-inner">
356 <div class="footer-left">
357 <a href="/" class="footer-logo">AUDIT<span>LABS</span></a>
358 <p>© 2026 Audit Labs — an independent, open-source project built for the community.</p>
359 <p class="footer-note">Illustrative example. “Northwind” and its findings are fictional. Framework mappings are the maintainers’ own interpretation — a failing row is evidence, not a compliance verdict.</p>
360 </div>
361 <div class="footer-links">
362 <a href="/">Home</a>
363 <a href="https://github.com/audit-labs">GitHub</a>
364 </div>
365 </div>
366</footer>
367
368</body>
369</html>