audit-labs/audit-report

Turn audit-tools evidence packages into control-mapped, auditor-ready reports. audit compliance evidence reporting https://audit-labs.dev/audit-report/

Commit 250db17024

250db1702432770222690977496980122a4318de

parent: 92ba96f545

Verified · cmc

cmc <hello@cleberg.net> · 2026-08-06 07:32 UTC 

docs: wire CI examples to audit-tools' exact invocation

audit-tools ships per-platform scripts, not a package, so the workflows now
check out / clone the repo, install its requirements.txt, and run
applications/<platform>/audit.py with the real flags and env vars: GitHub uses
--org with GITHUB_TOKEN/GITHUB_ORG; GitLab uses --group/--url defaulted to
CI_PROJECT_ROOT_NAMESPACE and CI_API_V4_URL. docs/ci.md gains the exact
per-platform collection commands for AWS, GitHub, and GitLab.

Layout: unified · split

docs/ci.md +23 −4
@@ -12,10 +12,29 @@ Ready-to-copy starting points:
12- [`examples/github-actions-audit.yml`](../examples/github-actions-audit.yml) 12- [`examples/github-actions-audit.yml`](../examples/github-actions-audit.yml)
13- [`examples/gitlab-ci-audit.yml`](../examples/gitlab-ci-audit.yml) 13- [`examples/gitlab-ci-audit.yml`](../examples/gitlab-ci-audit.yml)
14 14
15> The collection step in the examples shows both a module entrypoint 15> `audit-tools` is not on PyPI — it is a set of per-platform scripts. Check the
16> (`python -m audit_tools.github`) and a script fallback (`python audit.py`). 16> repo out (or clone it) and run `applications/<platform>/audit.py` directly;
17> Use whichever your installed `audit-tools` exposes; everything downstream only 17> Python puts the script's own directory on `sys.path`, so no `cd` is needed.
18> needs the `./output/<platform>_audit_<subject>_<date>/` directory it writes. 18> Pass an absolute `--out` so every platform's package lands in one folder.
19> Everything downstream only needs the
20> `<out>/<platform>_audit_<subject>_<date>/` directory it writes.
21
22## Exact collection commands
23
24Install `audit-tools`' dependencies once (`pip install -r audit-tools/requirements.txt`),
25export the platform's credentials, then:
26
27```bash
28# AWS — credentials come from the standard AWS chain (env vars, profile, OIDC role)
29python audit-tools/applications/aws/audit.py --out "$PWD/output"
30
31# GitHub — needs GITHUB_TOKEN (read-only) in the environment
32python audit-tools/applications/github/audit.py --org "$ORG" --out "$PWD/output"
33
34# GitLab — needs GITLAB_TOKEN (read_api) in the environment
35python audit-tools/applications/gitlab/audit.py \
36 --group "$GROUP" --url "$API_V4_URL" --out "$PWD/output"
37```
19 38
20## Gating strategies 39## Gating strategies
21 40
examples/github-actions-audit.yml +25 −17
@@ -1,12 +1,18 @@
1# Example GitHub Actions workflow: collect evidence, then report on it. 1# Example GitHub Actions workflow: collect evidence, then report on it.
2# 2#
3# Copy into .github/workflows/audit.yml in the repository you want to audit and 3# Copy into .github/workflows/audit.yml in a repository owned by the org you
4# adjust the collection step to your platform. It runs on a schedule and on 4# want to audit and adjust as needed. It runs on a schedule and on demand,
5# demand, produces a control-mapped report, and fails the run if a high-severity 5# produces a control-mapped report, and fails the run if any high-severity
6# control regresses against the previous package committed to the repo. 6# control is unsupported.
7# 7#
8# Requires two org/repo secrets for the GitHub collector: AUDIT_GITHUB_TOKEN 8# audit-tools is not published to PyPI — it is a set of per-platform scripts, so
9# (a read-only token for the org you audit) and the org name in AUDIT_ORG. 9# this checks the repo out and runs applications/github/audit.py directly.
10# audit-report *is* pip-installable from git.
11#
12# Required repository/org secret:
13# AUDIT_GITHUB_TOKEN a read-only token with org + security-events scope
14# Required variable (Settings > Variables), or hard-code below:
15# AUDIT_ORG the organization login to audit
10 16
11name: compliance-evidence 17name: compliance-evidence
12 18
@@ -22,32 +28,34 @@ jobs:
22 audit: 28 audit:
23 runs-on: ubuntu-latest 29 runs-on: ubuntu-latest
24 steps: 30 steps:
25 - uses: actions/checkout@v4 31 - name: Check out audit-tools (the collector)
32 uses: actions/checkout@v4
33 with:
34 repository: audit-labs/audit-tools
35 path: audit-tools
26 36
27 - uses: actions/setup-python@v5 37 - uses: actions/setup-python@v5
28 with: 38 with:
29 python-version: "3.12" 39 python-version: "3.12"
30 40
31 - name: Install tools 41 - name: Install collector deps and the reporter
32 run: | 42 run: |
33 python -m pip install --upgrade pip 43 python -m pip install --upgrade pip
34 # The reporter: 44 pip install -r audit-tools/requirements.txt
35 pip install "audit-report @ git+https://github.com/audit-labs/audit-report" 45 pip install "audit-report @ git+https://github.com/audit-labs/audit-report"
36 # The collector (audit-tools ships CLIs per platform):
37 pip install "audit-tools @ git+https://github.com/audit-labs/audit-tools"
38 46
39 - name: Collect evidence (GitHub example) 47 - name: Collect GitHub evidence
40 env: 48 env:
41 GITHUB_TOKEN: ${{ secrets.AUDIT_GITHUB_TOKEN }} 49 GITHUB_TOKEN: ${{ secrets.AUDIT_GITHUB_TOKEN }}
42 GITHUB_ORG: ${{ secrets.AUDIT_ORG }} 50 GITHUB_ORG: ${{ vars.AUDIT_ORG }}
43 run: | 51 run: |
44 # Produces ./output/github_audit_<org>_<date>/ 52 # Writes $GITHUB_WORKSPACE/output/github_audit_<org>_<date>/
45 python -m audit_tools.github --out ./output || \ 53 python audit-tools/applications/github/audit.py \
46 python audit.py --out ./output # fall back to the script entrypoint 54 --org "$GITHUB_ORG" --out "$GITHUB_WORKSPACE/output"
47 55
48 - name: Locate the newest package 56 - name: Locate the newest package
49 id: pkg 57 id: pkg
50 run: echo "dir=$(ls -d ./output/*_audit_* | sort | tail -n1)" >> "$GITHUB_OUTPUT" 58 run: echo "dir=$(ls -d "$GITHUB_WORKSPACE"/output/*_audit_* | sort | tail -n1)" >> "$GITHUB_OUTPUT"
51 59
52 - name: Generate evidence report 60 - name: Generate evidence report
53 run: | 61 run: |
examples/gitlab-ci-audit.yml +22 −9
@@ -1,11 +1,17 @@
1# Example GitLab CI configuration: collect evidence, then report on it. 1# Example GitLab CI configuration: collect evidence, then report on it.
2# 2#
3# Copy into .gitlab-ci.yml (or include it) in the project you want to audit. 3# Copy into .gitlab-ci.yml (or `include:` it) in a project under the group you
4# It produces a control-mapped report as a job artifact and fails the pipeline 4# want to audit. It produces a control-mapped report as a job artifact and fails
5# if any high-severity control is not supported. 5# the pipeline if any high-severity control is unsupported.
6# 6#
7# Set CI/CD variables GITLAB_TOKEN (read-only) and GITLAB_GROUP for the group 7# audit-tools is not published to PyPI — it is a set of per-platform scripts, so
8# you audit. 8# this clones the repo and runs applications/gitlab/audit.py directly.
9# audit-report *is* pip-installable from git.
10#
11# Required CI/CD variable (Settings > CI/CD > Variables):
12# GITLAB_TOKEN a token with read_api scope for the group
13# GITLAB_GROUP defaults to this project's top-level group and the API URL to the
14# instance running the pipeline, so it audits "where it lives" out of the box.
9 15
10stages: [audit] 16stages: [audit]
11 17
@@ -17,13 +23,20 @@ compliance-evidence:
17 - if: $CI_PIPELINE_SOURCE == "web" # manual "Run pipeline" 23 - if: $CI_PIPELINE_SOURCE == "web" # manual "Run pipeline"
18 variables: 24 variables:
19 PIP_DISABLE_PIP_VERSION_CHECK: "1" 25 PIP_DISABLE_PIP_VERSION_CHECK: "1"
26 GITLAB_GROUP: $CI_PROJECT_ROOT_NAMESPACE
20 before_script: 27 before_script:
28 - apt-get update && apt-get install -y --no-install-recommends git
29 - git clone --depth 1 https://github.com/audit-labs/audit-tools.git
30 - pip install -r audit-tools/requirements.txt
21 - pip install "audit-report @ git+https://github.com/audit-labs/audit-report" 31 - pip install "audit-report @ git+https://github.com/audit-labs/audit-report"
22 - pip install "audit-tools @ git+https://github.com/audit-labs/audit-tools"
23 script: 32 script:
24 # Produces ./output/gitlab_audit_<group>_<date>/ 33 # Writes $CI_PROJECT_DIR/output/gitlab_audit_<group>_<date>/ against this
25 - python -m audit_tools.gitlab --out ./output || python audit.py --out ./output 34 # instance's API ($CI_API_V4_URL is provided automatically by GitLab).
26 - PKG=$(ls -d ./output/*_audit_* | sort | tail -n1) 35 - >
36 python audit-tools/applications/gitlab/audit.py
37 --group "$GITLAB_GROUP" --url "$CI_API_V4_URL"
38 --out "$CI_PROJECT_DIR/output"
39 - PKG=$(ls -d "$CI_PROJECT_DIR"/output/*_audit_* | sort | tail -n1)
27 - audit-report "$PKG" --format md,html,json --out ./report 40 - audit-report "$PKG" --format md,html,json --out ./report
28 - audit-report "$PKG" --fail-on high 41 - audit-report "$PKG" --fail-on high
29 artifacts: 42 artifacts: