audit-labs/audit-report
Turn audit-tools evidence packages into control-mapped, auditor-ready reports.
clone: git clone https://gitbay.org/audit-labs/audit-report.git
v0.1.0: README.md · raw
1# audit-report
2
3[](LICENSE)
4[]()
5
6Turns an [audit-tools](https://github.com/audit-labs/audit-tools) evidence
7package into a control-mapped, auditor-ready report — Markdown, self-contained
8HTML, or JSON — driven by declarative rulesets.
9
10`audit-tools` collects raw CSVs (IAM users, branch protections, security
11groups…). `audit-report` reads that package, applies pass/fail rules, maps each
12result to **SOC 2, ISO 27001, and NIST SP 800-53** controls, and produces a
13report you can hand to an auditor or gate a pipeline on.
14
15> Like [gh-attest](https://github.com/audit-labs/gh-attest), this tool produces
16> *evidence*, not a compliance verdict. A failing row means a setting is in a
17> state that does not support a control; the final judgment belongs to the
18> organization and its auditor.
19
20## Install
21
22```bash
23git clone https://github.com/audit-labs/audit-report
24cd audit-report
25python -m venv .venv && source .venv/bin/activate
26pip install -e .
27```
28
29## Usage
30
31Point it at a package directory produced by `audit-tools`:
32
33```bash
34# Markdown report to stdout (platform + ruleset auto-detected from the dir name)
35audit-report ./output/aws_audit_default_2026-07-29
36
37# All three formats into ./report/
38audit-report ./output/github_audit_acme_2026-07-29 --format md,html,json --out report/
39
40# Gate CI: exit non-zero if any high-severity check fails
41audit-report ./output/aws_audit_prod_2026-07-29 --fail-on high --out report/
42```
43
44### Diff mode — compare two packages
45
46Pass `--baseline` to report how a package **drifted** from an earlier one. Both
47are evaluated with the same ruleset; the report classifies each rule as
48regressed, fixed, drifted (an ongoing failure whose evidence rows changed),
49changed, or unchanged — and shows exactly which evidence rows appeared or
50disappeared.
51
52```bash
53# How did prod change between two audit runs?
54audit-report ./output/aws_audit_prod_2026-07-29 \
55 --baseline ./output/aws_audit_prod_2026-06-29 \
56 --format md,html --out drift/
57
58# Fail CI if this change regressed any high-severity control
59audit-report ./output/aws_audit_prod_2026-07-29 \
60 --baseline ./output/aws_audit_prod_2026-06-29 --fail-on high
61```
62
63In diff mode `--fail-on` gates on **regressions** at or above the given
64severity, and output files are named `diff.*` instead of `report.*`.
65
66### Trend mode — track controls over time
67
68Pass `--trend` and point at a **folder of dated packages** (for example
69audit-tools' `output/`). Every package in the series is evaluated with the same
70ruleset and laid out as a timeline — one row per rule, one column per date — so
71you can watch a control drift in and out of compliance.
72
73```bash
74# Heatmap of every control across all retained prod packages
75audit-report ./output --trend --format html,json --out trend/
76
77# Disambiguate when the folder holds more than one series
78audit-report ./output --trend --subject prod --out trend/
79```
80
81The series must share one platform and subject (use `--subject` to pick one) and
82contain at least two packages. In trend mode `--fail-on` reflects the **latest**
83package, so it can double as a snapshot gate. Output files are named `trend.*`.
84
85You can also run it without installing:
86
87```bash
88python -m audit_report ./output/aws_audit_default_2026-07-29
89```
90
91### Options
92
93| Flag | Description |
94| --- | --- |
95| `--baseline PATH` | Diff mode: report how `PACKAGE` drifted from this earlier package. |
96| `--trend` | Trend mode: treat `PACKAGE` as a folder of dated packages and chart each rule over time. |
97| `--subject NAME` | In trend mode, pick one subject when the folder holds several series. |
98| `--ruleset PATH` | Use a specific ruleset instead of the bundled one for the detected platform. |
99| `--format md,html,json` | One or more output formats (default: `md`). |
100| `--out DIR` | Write `report.<ext>` files into `DIR`. Without it, the first format prints to stdout. |
101| `--fail-on low\|medium\|high\|none` | Exit non-zero when a failing finding — or, in diff mode, a regression — meets this severity (default: `none`). |
102
103## What a report contains
104
105- **Summary** — failing / passing / not-applicable counts.
106- **Control coverage matrix** — every cited control, its framework, and a
107 worst-wins status rolled up from the rules that reference it. One failing rule
108 marks the control as not fully evidenced.
109- **Findings** — failures first, then by severity, each with the reason, why it
110 matters, remediation, and the exact evidence rows that failed.
111
112## Bundled rulesets
113
114| Platform | Package prefix | Checks include |
115| --- | --- | --- |
116| AWS | `aws_audit_*` | Console-user MFA, access-key rotation, root MFA & keys, password policy, open SSH, public S3, CloudTrail logging |
117| GitHub | `github_audit_*` | Org 2FA, base permission, secret-scanning push protection, default-branch protection, required reviews |
118| GitLab | `gitlab_audit_*` | Force-push on protected branches, code-owner approval, approval-rule strength, public projects, instance password policy, audit logging |
119
120## Writing your own rules
121
122A ruleset is a YAML file: a `platform` and a list of `rules`. Each rule names a
123CSV table, a check, the controls it maps to, and human-readable text. Point at
124one with `--ruleset`.
125
126```yaml
127platform: aws
128rules:
129 - id: aws.iam.console-mfa
130 title: Console users have MFA enabled
131 table: iam_users # <table>.csv inside the package
132 severity: high # high | medium | low
133 controls: [SOC2:CC6.1, ISO:A.5.17, NIST:IA-2]
134 rationale: A console user without a second factor is one stolen password from access.
135 remediation: Enforce MFA for all console users, or remove their console password.
136 check:
137 type: fail_rows_where # every matching row is a failure
138 when:
139 all:
140 - {column: console_password, op: is_true}
141 - {column: mfa_enabled, op: is_false}
142```
143
144**Check types**
145
146| Type | Meaning |
147| --- | --- |
148| `fail_rows_where` | Each row matching `when` is a failing finding. |
149| `fail_if_any_rows` | The presence of any row (optionally filtered by `when`) is a failure. |
150| `require_any_row` | Passes only if at least one row satisfies `when`. |
151| `assert_row` | Checks each condition in `require` against the first row of a single-row config table. |
152
153**Condition language** — a condition is a leaf `{column, op, value}`, or one of
154`{all: [...]}`, `{any: [...]}`, `{not: ...}`. Operators: `equals`, `not_equals`,
155`is_true`, `is_false`, `in`, `not_in`, `gt`, `gte`, `lt`, `lte`, `empty`,
156`not_empty`. Control codes referenced by a rule must exist in
157[`audit_report/catalog.py`](audit_report/catalog.py).
158
159## Continuous integration
160
161Run it on a schedule or in a pipeline to keep evidence current and gate on
162regressions. See [docs/ci.md](docs/ci.md) and the ready-to-copy examples:
163
164- [`examples/github-actions-audit.yml`](examples/github-actions-audit.yml)
165- [`examples/gitlab-ci-audit.yml`](examples/gitlab-ci-audit.yml)
166
167The `--fail-on` exit code (`1` = a finding/regression met the threshold, `2` =
168usage error) lets a workflow separate "the audit found a problem" from "the job
169is misconfigured".
170
171## Development
172
173```bash
174pip install -e ".[dev]"
175pytest # run from the project root
176ruff check .
177```
178
179## License
180
181GPL-3.0-or-later. See [LICENSE](LICENSE).