audit-labs/audit-report
clone: git clone https://gitbay.org/audit-labs/audit-report.git
main: README.md · raw
1# audit-report
2
3[](LICENSE)
4[]()
5
6Turns an [audit-tools](https://github.com/audit-labs/audit-tools) evidence
7package into a control-mapped, auditor-ready report — Markdown, self-contained
8HTML, or JSON — driven by declarative rulesets.
9
10`audit-tools` collects raw CSVs (IAM users, branch protections, security
11groups…). `audit-report` reads that package, applies pass/fail rules, maps each
12result to **SOC 2, ISO 27001, and NIST SP 800-53** controls, and produces a
13report you can hand to an auditor or gate a pipeline on.
14
15> Like [gh-attest](https://github.com/audit-labs/gh-attest), this tool produces
16> *evidence*, not a compliance verdict. A failing row means a setting is in a
17> state that does not support a control; the final judgment belongs to the
18> organization and its auditor.
19
20## Install
21
22```bash
23pip install audit-report
24```
25
26To hack on it from a clone instead, see [Development](#development).
27
28## Usage
29
30Point it at a package directory produced by `audit-tools`:
31
32```bash
33# Markdown report to stdout (platform + ruleset auto-detected from the dir name)
34audit-report ./output/aws_audit_default_2026-07-29
35
36# All three formats into ./report/
37audit-report ./output/github_audit_acme_2026-07-29 --format md,html,json --out report/
38
39# Gate CI: exit non-zero if any high-severity check fails
40audit-report ./output/aws_audit_prod_2026-07-29 --fail-on high --out report/
41```
42
43### Diff mode — compare two packages
44
45Pass `--baseline` to report how a package **drifted** from an earlier one. Both
46are evaluated with the same ruleset; the report classifies each rule as
47regressed, fixed, drifted (an ongoing failure whose evidence rows changed),
48changed, or unchanged — and shows exactly which evidence rows appeared or
49disappeared.
50
51```bash
52# How did prod change between two audit runs?
53audit-report ./output/aws_audit_prod_2026-07-29 \
54 --baseline ./output/aws_audit_prod_2026-06-29 \
55 --format md,html --out drift/
56
57# Fail CI if this change regressed any high-severity control
58audit-report ./output/aws_audit_prod_2026-07-29 \
59 --baseline ./output/aws_audit_prod_2026-06-29 --fail-on high
60```
61
62In diff mode `--fail-on` gates on **regressions** at or above the given
63severity, and output files are named `diff.*` instead of `report.*`.
64
65### Trend mode — track controls over time
66
67Pass `--trend` and point at a **folder of dated packages** (for example
68audit-tools' `output/`). Every package in the series is evaluated with the same
69ruleset and laid out as a timeline — one row per rule, one column per date — so
70you can watch a control drift in and out of compliance.
71
72```bash
73# Heatmap of every control across all retained prod packages
74audit-report ./output --trend --format html,json --out trend/
75
76# Disambiguate when the folder holds more than one series
77audit-report ./output --trend --subject prod --out trend/
78```
79
80The series must share one platform and subject (use `--subject` to pick one) and
81contain at least two packages. In trend mode `--fail-on` reflects the **latest**
82package, so it can double as a snapshot gate. Output files are named `trend.*`.
83
84You can also run it without installing:
85
86```bash
87python -m audit_report ./output/aws_audit_default_2026-07-29
88```
89
90### Options
91
92| Flag | Description |
93| --- | --- |
94| `--baseline PATH` | Diff mode: report how `PACKAGE` drifted from this earlier package. |
95| `--trend` | Trend mode: treat `PACKAGE` as a folder of dated packages and chart each rule over time. |
96| `--subject NAME` | In trend mode, pick one subject when the folder holds several series. |
97| `--ruleset PATH` | Use a specific ruleset instead of the bundled one for the detected platform. |
98| `--format md,html,json` | One or more output formats (default: `md`). |
99| `--out DIR` | Write `report.<ext>` files into `DIR`. Without it, the first format prints to stdout. |
100| `--fail-on low\|medium\|high\|none` | Exit non-zero when a failing finding — or, in diff mode, a regression — meets this severity (default: `none`). |
101
102## What a report contains
103
104- **Summary** — failing / passing / not-applicable counts.
105- **Control coverage matrix** — every cited control, its framework, and a
106 worst-wins status rolled up from the rules that reference it. One failing rule
107 marks the control as not fully evidenced.
108- **Findings** — failures first, then by severity, each with the reason, why it
109 matters, remediation, and the exact evidence rows that failed.
110
111## Bundled rulesets
112
113| Platform | Package prefix | Checks include |
114| --- | --- | --- |
115| AWS | `aws_audit_*` | Console-user MFA, access-key rotation, root MFA & keys, password policy, open SSH, public S3, CloudTrail logging |
116| GitHub | `github_audit_*` | Org 2FA, base permission, secret-scanning push protection, default-branch protection, required reviews |
117| GitLab | `gitlab_audit_*` | Force-push on protected branches, code-owner approval, approval-rule strength, public projects, instance password policy, audit logging |
118
119## Writing your own rules
120
121A ruleset is a YAML file: a `platform` and a list of `rules`. Each rule names a
122CSV table, a check, the controls it maps to, and human-readable text. Point at
123one with `--ruleset`.
124
125```yaml
126platform: aws
127rules:
128 - id: aws.iam.console-mfa
129 title: Console users have MFA enabled
130 table: iam_users # <table>.csv inside the package
131 severity: high # high | medium | low
132 controls: [SOC2:CC6.1, ISO:A.5.17, NIST:IA-2]
133 rationale: A console user without a second factor is one stolen password from access.
134 remediation: Enforce MFA for all console users, or remove their console password.
135 check:
136 type: fail_rows_where # every matching row is a failure
137 when:
138 all:
139 - {column: console_password, op: is_true}
140 - {column: mfa_enabled, op: is_false}
141```
142
143**Check types**
144
145| Type | Meaning |
146| --- | --- |
147| `fail_rows_where` | Each row matching `when` is a failing finding. |
148| `fail_if_any_rows` | The presence of any row (optionally filtered by `when`) is a failure. |
149| `require_any_row` | Passes only if at least one row satisfies `when`. |
150| `assert_row` | Checks each condition in `require` against the first row of a single-row config table. |
151
152**Condition language** — a condition is a leaf `{column, op, value}`, or one of
153`{all: [...]}`, `{any: [...]}`, `{not: ...}`. Operators: `equals`, `not_equals`,
154`is_true`, `is_false`, `in`, `not_in`, `gt`, `gte`, `lt`, `lte`, `empty`,
155`not_empty`. Control codes referenced by a rule must exist in
156[`audit_report/catalog.py`](audit_report/catalog.py).
157
158## Continuous integration
159
160Run it on a schedule or in a pipeline to keep evidence current and gate on
161regressions. See [docs/ci.md](docs/ci.md) and the ready-to-copy examples:
162
163- [`examples/github-actions-audit.yml`](examples/github-actions-audit.yml)
164- [`examples/gitlab-ci-audit.yml`](examples/gitlab-ci-audit.yml)
165
166The `--fail-on` exit code (`1` = a finding/regression met the threshold, `2` =
167usage error) lets a workflow separate "the audit found a problem" from "the job
168is misconfigured".
169
170## Stability
171
172`audit-report` is stable as of **v1.0.0** and follows [semantic versioning](https://semver.org).
173The JSON report schema — findings carrying their `controls` and
174`pass` / `fail` / `not_applicable` status — is a committed contract that
175[control-coverage](https://github.com/audit-labs/control-coverage) consumes
176directly; it will not break without a major-version bump.
177
178## Development
179
180```bash
181pip install -e ".[dev]"
182pytest # run from the project root
183ruff check .
184```
185
186## License
187
188GPL-3.0-or-later. See [LICENSE](LICENSE).