Commit `4187ceff35`

4187ceff35942b36fdbf3b3f520dfd92c99c003d

Verified · cmc

cmc <hello@cleberg.net> · 2026-08-07 01:07 UTC

Initial commit: control-coverage

Control-first coverage and blind-spot analysis over an evidence corpus.
Where the rest of the audit-labs toolchain is evidence-first (map each
collected finding to the controls it touches), control-coverage starts from
the complete catalog of a framework's controls and scores the evidence
against it — reporting a true coverage percentage and, more usefully, the
in-scope controls no finding touches.

Consumes audit-report JSON reports as a corpus. Bundles complete catalogs:
SOC 2 (all five Trust Services categories, 61 controls), ISO/IEC 27001:2022
Annex A (93), and the NIST SP 800-53 Rev. 5 moderate baseline (177 base
controls).

Analyses:
- coverage — per-control assurance state, blind spots, Statement of
  Applicability (md/html/json/soa)
- trend — how coverage moved between two corpora (md/html/json)
- crosswalk — evidence leverage across frameworks and the minimal evidence
  set via greedy set cover (md/html/json)

Scope files mark per-control and per-family exclusions with mandatory
justifications. CI gates: --fail-under and --fail-on-regression.

Layout: unified · split

.gitignore added +20

@@ -0,0 +1,20 @@
	1	.venv
	2	venv
	3
	4	# Python
	5	__pycache__/
	6	**/__pycache__/
	7	*.py[cod]
	8	*.egg-info/
	9	.pytest_cache/
	10	.ruff_cache/
	11	build/
	12	dist/
	13
	14	# Generated output
	15	/coverage-out/
	16	soa.md
	17	soa.html
	18	coverage.md
	19	coverage.html
	20	coverage.json

CODEOWNERS added +1

	@@ -0,0 +1 @@
		1	* @ccleberg @ekraai2

GUIDE.md added +198

@@ -0,0 +1,198 @@
	1	# Using control-coverage
	2
	3	A practical, task-oriented guide. For the conceptual overview see the
	4	[README](README.md); this walks through actually running the tool.
	5
	6	## The one thing to understand first
	7
	8	Every other Audit Labs tool is evidence-first: it starts from what you collected
	9	and tells you what it maps to. `control-coverage` is control-first: it starts
	10	from the complete list of a framework's controls and tells you how much of it your
	11	evidence addresses — and, more usefully, what it doesn't.
	12
	13	So the input is your evidence, and the output is measured against a fixed yardstick
	14	(the framework catalog) you didn't have to write.
	15
	16	## 1. Get the input: audit-report JSON
	17
	18	The corpus is one or more JSON reports from `audit-report`. Produce them with its
	19	`--format json` flag, one per platform:
	20
	21	```bash
	22	audit-report ./output/aws_audit_prod_2026-02-01 --format json --out reports/
	23	audit-report ./output/github_audit_prod_2026-02-01 --format json --out reports/
	24	```
	25
	26	You now have `reports/.json`. That directory is* a corpus — coverage aggregates
	27	every report in it into one per-framework picture, so AWS, GitHub, and GitLab
	28	evidence all count toward the same SOC 2 number.
	29
	30	> No audit-report packages yet? Any JSON with the same shape works — a list of
	31	> `findings`, each with `controls: ["SOC2:CC6.1", ...]` and a `status` of `pass`,
	32	> `fail`, or `not_applicable`.
	33
	34	## 2. Install
	35
	36	```bash
	37	git clone https://github.com/audit-labs/control-coverage
	38	cd control-coverage
	39	python -m venv .venv && source .venv/bin/activate
	40	pip install -e .
	41	```
	42
	43	## 3. The four things you'll actually do
	44
	45	### A. "How covered am I, and what am I missing?"
	46
	47	```bash
	48	control-coverage reports/
	49	```
	50
	51	Prints a Markdown report: a per-framework summary table, then the blind spots
	52	(in-scope controls no finding touches), then the full matrix. Frameworks are inferred
	53	from the codes your corpus cites.
	54
	55	Want just the gap list, nothing else?
	56
	57	```bash
	58	control-coverage reports/ --framework SOC2 --blind-spots
	59	```
	60
	61	Want files to hand off? Write all formats to a directory:
	62
	63	```bash
	64	control-coverage reports/ --format md,html,json --out coverage-out/
	65	```
	66
	67	- md — human-readable, good for a PR comment or a wiki paste.
	68	- html — self-contained, printable, has coverage bars. Attach to a workpaper.
	69	- json — for dashboards or further scripting.
	70
	71	### B. "Produce a Statement of Applicability"
	72
	73	First write a scope file. It selects frameworks and records exclusions — each with a
	74	mandatory reason (an unjustified exclusion is rejected):
	75
	76	```yaml
	77	# soa.yaml
	78	subject: Acme Production
	79	frameworks: [SOC2, ISO]
	80	exclusions:
	81	- control: ISO:A.7.1
	82	reason: "Fully cloud-hosted; no physical premises are in scope for the ISMS."
	83	- control: ISO:A.5.7
	84	reason: "No formal threat-intelligence program; risk accepted by the CISO for 2026."
	85	exclude_families:
	86	- {framework: SOC2, family: Privacy, reason: "Privacy category not in the SOC 2 audit scope."}
	87	owners:
	88	SOC2:CC6.1: platform-team
	89	```
	90
	91	`exclusions` drops one control; `exclude_families` drops a whole category, ISO theme,
	92	or NIST family at once (how audit scope is really decided). Both require a reason.
	93
	94	Then generate coverage over the in-scope controls, plus the SoA itself:
	95
	96	```bash
	97	control-coverage reports/ --scope soa.yaml --format md,soa --out coverage-out/
	98	```
	99
	100	`coverage-out/soa.md` lists every control, whether it applies, its implementation
	101	status (derived from your evidence, not asserted by hand), and the justification.
	102	Excluded controls are recorded, not counted as gaps.
	103
	104	### C. "What changed since last time?"
	105
	106	Keep last month's reports around. Point `--baseline` at them:
	107
	108	```bash
	109	control-coverage reports/2026-02/ --baseline reports/2026-01/ --framework SOC2
	110	```
	111
	112	You get a movement report:
	113
	114	- improved — a control got more assurance (e.g. failing → supported).
	115	- regressed — a control lost assurance (e.g. supported → failing).
	116	- gained — a blind spot became addressed (coverage went up).
	117	- lost — an addressed control became a blind spot (coverage went down).
	118
	119	`--baseline` accepts a single file or a directory.
	120
	121	### D. "Which evidence is doing the most work?"
	122
	123	```bash
	124	control-coverage reports/ --framework SOC2,ISO,NIST --crosswalk
	125	```
	126
	127	Two things come out:
	128
	129	- Evidence leverage — each check and the controls it supports, across all three
	130	frameworks. You'll see that one 2FA check earns SOC 2 CC6.1 + ISO A.5.17 + NIST IA-2.
	131	- Minimal evidence set — the fewest checks that still cover every addressed
	132	control. This is your walkthrough/sampling short-list: pull these and you've touched
	133	everything the full corpus touches.
	134
	135	## 4. Reading the numbers
	136
	137	Every in-scope control is in exactly one state:
	138
	139	\| State \| What it means \| Counts toward… \|
	140	\| --- \| --- \| --- \|
	141	\| supported \| Something passes here, nothing fails \| coverage and assured \|
	142	\| failing \| Something fails here (worst wins) \| coverage \|
	143	\| asserted \| Mapped, but the data was absent \| coverage \|
	144	\| unaddressed \| Nothing maps here — a blind spot \| neither \|
	145	\| out of scope \| Excluded in the scope file, with a reason \| neither (removed from the denominator) \|
	146
	147	- Coverage % = supported + failing + asserted, over in-scope controls. *"How much
	148	of the framework am I even looking at?"*
	149	- Assured % = supported only, over in-scope controls. *"How much do I have good
	150	evidence for?"*
	151
	152	A low coverage number on a fresh corpus is expected — the framework is large and your
	153	automated checks touch a slice of it. The value is knowing exactly which slice, and
	154	watching coverage climb (via `--baseline`) as you add evidence.
	155
	156	## 5. Wire it into CI
	157
	158	Two independent gates, both exit non-zero to fail a build:
	159
	160	```bash
	161	# Fail if any framework's coverage drops below a floor
	162	control-coverage reports/ --scope soa.yaml --fail-under 60
	163
	164	# Fail if anything regressed or lost coverage versus the last run
	165	control-coverage reports/ --baseline last-run/ --fail-on-regression
	166	```
	167
	168	See [`examples/github-actions-coverage.yml`](examples/github-actions-coverage.yml) for
	169	a scheduled workflow that runs both and uploads the reports as artifacts.
	170
	171	## 6. Frameworks and codes
	172
	173	\| Framework \| Pass as \| Catalog \|
	174	\| --- \| --- \| --- \|
	175	\| SOC 2 Trust Services Criteria \| `SOC2` \| All five categories, 61 controls \|
	176	\| ISO/IEC 27001:2022 Annex A \| `ISO` (or `iso27001`) \| All 93 Annex A controls \|
	177	\| NIST SP 800-53 Rev. 5 \| `NIST` (or `800-53`) \| Moderate baseline, 177 base controls \|
	178
	179	Control codes are `FRAMEWORK:ID` — `SOC2:CC6.1`, `ISO:A.5.17`, `NIST:IA-2` — the same
	180	codes `audit-report` rulesets already emit, so the two tools line up with no
	181	translation. If your corpus cites a code whose framework is loaded but the catalog
	182	doesn't define it (a typo or a renamed control), it's reported under **Unmatched
	183	control codes** rather than silently dropped.
	184
	185	## Gotchas
	186
	187	- `--crosswalk` and `--baseline` can't be combined — they're different analyses.
	188	- Trend and crosswalk emit `md`, `html`, and `json` (not `soa`). Coverage emits all four.
	189	- SOC 2 defaults to all five categories (61 controls). Most reports scope only some.
	190	Drop the ones you're not audited on with `exclude_families` (see §2) so coverage
	191	reflects your real perimeter — e.g. exclude `Privacy` and `Processing Integrity`.
	192	- NIST coverage looks low because the moderate baseline is large (177 controls) and
	193	most are organizational/physical/personnel controls no automated scanner evidences.
	194	That's the point — those are your blind spots. Use a scope file to exclude the ones
	195	handled by policy rather than tooling, so the number reflects your real perimeter.
	196	- *An unaddressed control is a gap in evidence, not proof of a gap in controls.*
	197	It may just mean the signal isn't collected yet. The tool produces evidence, never a
	198	verdict.

LICENSE added +674

@@ -0,0 +1,674 @@
	1	GNU GENERAL PUBLIC LICENSE
	2	Version 3, 29 June 2007
	3
	4	Copyright (C) 2007 Free Software Foundation, Inc. <https://fsf.org/>
	5	Everyone is permitted to copy and distribute verbatim copies
	6	of this license document, but changing it is not allowed.
	7
	8	Preamble
	9
	10	The GNU General Public License is a free, copyleft license for
	11	software and other kinds of works.
	12
	13	The licenses for most software and other practical works are designed
	14	to take away your freedom to share and change the works. By contrast,
	15	the GNU General Public License is intended to guarantee your freedom to
	16	share and change all versions of a program--to make sure it remains free
	17	software for all its users. We, the Free Software Foundation, use the
	18	GNU General Public License for most of our software; it applies also to
	19	any other work released this way by its authors. You can apply it to
	20	your programs, too.
	21
	22	When we speak of free software, we are referring to freedom, not
	23	price. Our General Public Licenses are designed to make sure that you
	24	have the freedom to distribute copies of free software (and charge for
	25	them if you wish), that you receive source code or can get it if you
	26	want it, that you can change the software or use pieces of it in new
	27	free programs, and that you know you can do these things.
	28
	29	To protect your rights, we need to prevent others from denying you
	30	these rights or asking you to surrender the rights. Therefore, you have
	31	certain responsibilities if you distribute copies of the software, or if
	32	you modify it: responsibilities to respect the freedom of others.
	33
	34	For example, if you distribute copies of such a program, whether
	35	gratis or for a fee, you must pass on to the recipients the same
	36	freedoms that you received. You must make sure that they, too, receive
	37	or can get the source code. And you must show them these terms so they
	38	know their rights.
	39
	40	Developers that use the GNU GPL protect your rights with two steps:
	41	(1) assert copyright on the software, and (2) offer you this License
	42	giving you legal permission to copy, distribute and/or modify it.
	43
	44	For the developers' and authors' protection, the GPL clearly explains
	45	that there is no warranty for this free software. For both users' and
	46	authors' sake, the GPL requires that modified versions be marked as
	47	changed, so that their problems will not be attributed erroneously to
	48	authors of previous versions.
	49
	50	Some devices are designed to deny users access to install or run
	51	modified versions of the software inside them, although the manufacturer
	52	can do so. This is fundamentally incompatible with the aim of
	53	protecting users' freedom to change the software. The systematic
	54	pattern of such abuse occurs in the area of products for individuals to
	55	use, which is precisely where it is most unacceptable. Therefore, we
	56	have designed this version of the GPL to prohibit the practice for those
	57	products. If such problems arise substantially in other domains, we
	58	stand ready to extend this provision to those domains in future versions
	59	of the GPL, as needed to protect the freedom of users.
	60
	61	Finally, every program is threatened constantly by software patents.
	62	States should not allow patents to restrict development and use of
	63	software on general-purpose computers, but in those that do, we wish to
	64	avoid the special danger that patents applied to a free program could
	65	make it effectively proprietary. To prevent this, the GPL assures that
	66	patents cannot be used to render the program non-free.
	67
	68	The precise terms and conditions for copying, distribution and
	69	modification follow.
	70
	71	TERMS AND CONDITIONS
	72
	73	0. Definitions.
	74
	75	"This License" refers to version 3 of the GNU General Public License.
	76
	77	"Copyright" also means copyright-like laws that apply to other kinds of
	78	works, such as semiconductor masks.
	79
	80	"The Program" refers to any copyrightable work licensed under this
	81	License. Each licensee is addressed as "you". "Licensees" and
	82	"recipients" may be individuals or organizations.
	83
	84	To "modify" a work means to copy from or adapt all or part of the work
	85	in a fashion requiring copyright permission, other than the making of an
	86	exact copy. The resulting work is called a "modified version" of the
	87	earlier work or a work "based on" the earlier work.
	88
	89	A "covered work" means either the unmodified Program or a work based
	90	on the Program.
	91
	92	To "propagate" a work means to do anything with it that, without
	93	permission, would make you directly or secondarily liable for
	94	infringement under applicable copyright law, except executing it on a
	95	computer or modifying a private copy. Propagation includes copying,
	96	distribution (with or without modification), making available to the
	97	public, and in some countries other activities as well.
	98
	99	To "convey" a work means any kind of propagation that enables other
	100	parties to make or receive copies. Mere interaction with a user through
	101	a computer network, with no transfer of a copy, is not conveying.
	102
	103	An interactive user interface displays "Appropriate Legal Notices"
	104	to the extent that it includes a convenient and prominently visible
	105	feature that (1) displays an appropriate copyright notice, and (2)
	106	tells the user that there is no warranty for the work (except to the
	107	extent that warranties are provided), that licensees may convey the
	108	work under this License, and how to view a copy of this License. If
	109	the interface presents a list of user commands or options, such as a
	110	menu, a prominent item in the list meets this criterion.
	111
	112	1. Source Code.
	113
	114	The "source code" for a work means the preferred form of the work
	115	for making modifications to it. "Object code" means any non-source
	116	form of a work.
	117
	118	A "Standard Interface" means an interface that either is an official
	119	standard defined by a recognized standards body, or, in the case of
	120	interfaces specified for a particular programming language, one that
	121	is widely used among developers working in that language.
	122
	123	The "System Libraries" of an executable work include anything, other
	124	than the work as a whole, that (a) is included in the normal form of
	125	packaging a Major Component, but which is not part of that Major
	126	Component, and (b) serves only to enable use of the work with that
	127	Major Component, or to implement a Standard Interface for which an
	128	implementation is available to the public in source code form. A
	129	"Major Component", in this context, means a major essential component
	130	(kernel, window system, and so on) of the specific operating system
	131	(if any) on which the executable work runs, or a compiler used to
	132	produce the work, or an object code interpreter used to run it.
	133
	134	The "Corresponding Source" for a work in object code form means all
	135	the source code needed to generate, install, and (for an executable
	136	work) run the object code and to modify the work, including scripts to
	137	control those activities. However, it does not include the work's
	138	System Libraries, or general-purpose tools or generally available free
	139	programs which are used unmodified in performing those activities but
	140	which are not part of the work. For example, Corresponding Source
	141	includes interface definition files associated with source files for
	142	the work, and the source code for shared libraries and dynamically
	143	linked subprograms that the work is specifically designed to require,
	144	such as by intimate data communication or control flow between those
	145	subprograms and other parts of the work.
	146
	147	The Corresponding Source need not include anything that users
	148	can regenerate automatically from other parts of the Corresponding
	149	Source.
	150
	151	The Corresponding Source for a work in source code form is that
	152	same work.
	153
	154	2. Basic Permissions.
	155
	156	All rights granted under this License are granted for the term of
	157	copyright on the Program, and are irrevocable provided the stated
	158	conditions are met. This License explicitly affirms your unlimited
	159	permission to run the unmodified Program. The output from running a
	160	covered work is covered by this License only if the output, given its
	161	content, constitutes a covered work. This License acknowledges your
	162	rights of fair use or other equivalent, as provided by copyright law.
	163
	164	You may make, run and propagate covered works that you do not
	165	convey, without conditions so long as your license otherwise remains
	166	in force. You may convey covered works to others for the sole purpose
	167	of having them make modifications exclusively for you, or provide you
	168	with facilities for running those works, provided that you comply with
	169	the terms of this License in conveying all material for which you do
	170	not control copyright. Those thus making or running the covered works
	171	for you must do so exclusively on your behalf, under your direction
	172	and control, on terms that prohibit them from making any copies of
	173	your copyrighted material outside their relationship with you.
	174
	175	Conveying under any other circumstances is permitted solely under
	176	the conditions stated below. Sublicensing is not allowed; section 10
	177	makes it unnecessary.
	178
	179	3. Protecting Users' Legal Rights From Anti-Circumvention Law.
	180
	181	No covered work shall be deemed part of an effective technological
	182	measure under any applicable law fulfilling obligations under article
	183	11 of the WIPO copyright treaty adopted on 20 December 1996, or
	184	similar laws prohibiting or restricting circumvention of such
	185	measures.
	186
	187	When you convey a covered work, you waive any legal power to forbid
	188	circumvention of technological measures to the extent such circumvention
	189	is effected by exercising rights under this License with respect to
	190	the covered work, and you disclaim any intention to limit operation or
	191	modification of the work as a means of enforcing, against the work's
	192	users, your or third parties' legal rights to forbid circumvention of
	193	technological measures.
	194
	195	4. Conveying Verbatim Copies.
	196
	197	You may convey verbatim copies of the Program's source code as you
	198	receive it, in any medium, provided that you conspicuously and
	199	appropriately publish on each copy an appropriate copyright notice;
	200	keep intact all notices stating that this License and any
	201	non-permissive terms added in accord with section 7 apply to the code;
	202	keep intact all notices of the absence of any warranty; and give all
	203	recipients a copy of this License along with the Program.
	204
	205	You may charge any price or no price for each copy that you convey,
	206	and you may offer support or warranty protection for a fee.
	207
	208	5. Conveying Modified Source Versions.
	209
	210	You may convey a work based on the Program, or the modifications to
	211	produce it from the Program, in the form of source code under the
	212	terms of section 4, provided that you also meet all of these conditions:
	213
	214	a) The work must carry prominent notices stating that you modified
	215	it, and giving a relevant date.
	216
	217	b) The work must carry prominent notices stating that it is
	218	released under this License and any conditions added under section
	219	7. This requirement modifies the requirement in section 4 to
	220	"keep intact all notices".
	221
	222	c) You must license the entire work, as a whole, under this
	223	License to anyone who comes into possession of a copy. This
	224	License will therefore apply, along with any applicable section 7
	225	additional terms, to the whole of the work, and all its parts,
	226	regardless of how they are packaged. This License gives no
	227	permission to license the work in any other way, but it does not
	228	invalidate such permission if you have separately received it.
	229
	230	d) If the work has interactive user interfaces, each must display
	231	Appropriate Legal Notices; however, if the Program has interactive
	232	interfaces that do not display Appropriate Legal Notices, your
	233	work need not make them do so.
	234
	235	A compilation of a covered work with other separate and independent
	236	works, which are not by their nature extensions of the covered work,
	237	and which are not combined with it such as to form a larger program,
	238	in or on a volume of a storage or distribution medium, is called an
	239	"aggregate" if the compilation and its resulting copyright are not
	240	used to limit the access or legal rights of the compilation's users
	241	beyond what the individual works permit. Inclusion of a covered work
	242	in an aggregate does not cause this License to apply to the other
	243	parts of the aggregate.
	244
	245	6. Conveying Non-Source Forms.
	246
	247	You may convey a covered work in object code form under the terms
	248	of sections 4 and 5, provided that you also convey the
	249	machine-readable Corresponding Source under the terms of this License,
	250	in one of these ways:
	251
	252	a) Convey the object code in, or embodied in, a physical product
	253	(including a physical distribution medium), accompanied by the
	254	Corresponding Source fixed on a durable physical medium
	255	customarily used for software interchange.
	256
	257	b) Convey the object code in, or embodied in, a physical product
	258	(including a physical distribution medium), accompanied by a
	259	written offer, valid for at least three years and valid for as
	260	long as you offer spare parts or customer support for that product
	261	model, to give anyone who possesses the object code either (1) a
	262	copy of the Corresponding Source for all the software in the
	263	product that is covered by this License, on a durable physical
	264	medium customarily used for software interchange, for a price no
	265	more than your reasonable cost of physically performing this
	266	conveying of source, or (2) access to copy the
	267	Corresponding Source from a network server at no charge.
	268
	269	c) Convey individual copies of the object code with a copy of the
	270	written offer to provide the Corresponding Source. This
	271	alternative is allowed only occasionally and noncommercially, and
	272	only if you received the object code with such an offer, in accord
	273	with subsection 6b.
	274
	275	d) Convey the object code by offering access from a designated
	276	place (gratis or for a charge), and offer equivalent access to the
	277	Corresponding Source in the same way through the same place at no
	278	further charge. You need not require recipients to copy the
	279	Corresponding Source along with the object code. If the place to
	280	copy the object code is a network server, the Corresponding Source
	281	may be on a different server (operated by you or a third party)
	282	that supports equivalent copying facilities, provided you maintain
	283	clear directions next to the object code saying where to find the
	284	Corresponding Source. Regardless of what server hosts the
	285	Corresponding Source, you remain obligated to ensure that it is
	286	available for as long as needed to satisfy these requirements.
	287
	288	e) Convey the object code using peer-to-peer transmission, provided
	289	you inform other peers where the object code and Corresponding
	290	Source of the work are being offered to the general public at no
	291	charge under subsection 6d.
	292
	293	A separable portion of the object code, whose source code is excluded
	294	from the Corresponding Source as a System Library, need not be
	295	included in conveying the object code work.
	296
	297	A "User Product" is either (1) a "consumer product", which means any
	298	tangible personal property which is normally used for personal, family,
	299	or household purposes, or (2) anything designed or sold for incorporation
	300	into a dwelling. In determining whether a product is a consumer product,
	301	doubtful cases shall be resolved in favor of coverage. For a particular
	302	product received by a particular user, "normally used" refers to a
	303	typical or common use of that class of product, regardless of the status
	304	of the particular user or of the way in which the particular user
	305	actually uses, or expects or is expected to use, the product. A product
	306	is a consumer product regardless of whether the product has substantial
	307	commercial, industrial or non-consumer uses, unless such uses represent
	308	the only significant mode of use of the product.
	309
	310	"Installation Information" for a User Product means any methods,
	311	procedures, authorization keys, or other information required to install
	312	and execute modified versions of a covered work in that User Product from
	313	a modified version of its Corresponding Source. The information must
	314	suffice to ensure that the continued functioning of the modified object
	315	code is in no case prevented or interfered with solely because
	316	modification has been made.
	317
	318	If you convey an object code work under this section in, or with, or
	319	specifically for use in, a User Product, and the conveying occurs as
	320	part of a transaction in which the right of possession and use of the
	321	User Product is transferred to the recipient in perpetuity or for a
	322	fixed term (regardless of how the transaction is characterized), the
	323	Corresponding Source conveyed under this section must be accompanied
	324	by the Installation Information. But this requirement does not apply
	325	if neither you nor any third party retains the ability to install
	326	modified object code on the User Product (for example, the work has
	327	been installed in ROM).
	328
	329	The requirement to provide Installation Information does not include a
	330	requirement to continue to provide support service, warranty, or updates
	331	for a work that has been modified or installed by the recipient, or for
	332	the User Product in which it has been modified or installed. Access to a
	333	network may be denied when the modification itself materially and
	334	adversely affects the operation of the network or violates the rules and
	335	protocols for communication across the network.
	336
	337	Corresponding Source conveyed, and Installation Information provided,
	338	in accord with this section must be in a format that is publicly
	339	documented (and with an implementation available to the public in
	340	source code form), and must require no special password or key for
	341	unpacking, reading or copying.
	342
	343	7. Additional Terms.
	344
	345	"Additional permissions" are terms that supplement the terms of this
	346	License by making exceptions from one or more of its conditions.
	347	Additional permissions that are applicable to the entire Program shall
	348	be treated as though they were included in this License, to the extent
	349	that they are valid under applicable law. If additional permissions
	350	apply only to part of the Program, that part may be used separately
	351	under those permissions, but the entire Program remains governed by
	352	this License without regard to the additional permissions.
	353
	354	When you convey a copy of a covered work, you may at your option
	355	remove any additional permissions from that copy, or from any part of
	356	it. (Additional permissions may be written to require their own
	357	removal in certain cases when you modify the work.) You may place
	358	additional permissions on material, added by you to a covered work,
	359	for which you have or can give appropriate copyright permission.
	360
	361	Notwithstanding any other provision of this License, for material you
	362	add to a covered work, you may (if authorized by the copyright holders of
	363	that material) supplement the terms of this License with terms:
	364
	365	a) Disclaiming warranty or limiting liability differently from the
	366	terms of sections 15 and 16 of this License; or
	367
	368	b) Requiring preservation of specified reasonable legal notices or
	369	author attributions in that material or in the Appropriate Legal
	370	Notices displayed by works containing it; or
	371
	372	c) Prohibiting misrepresentation of the origin of that material, or
	373	requiring that modified versions of such material be marked in
	374	reasonable ways as different from the original version; or
	375
	376	d) Limiting the use for publicity purposes of names of licensors or
	377	authors of the material; or
	378
	379	e) Declining to grant rights under trademark law for use of some
	380	trade names, trademarks, or service marks; or
	381
	382	f) Requiring indemnification of licensors and authors of that
	383	material by anyone who conveys the material (or modified versions of
	384	it) with contractual assumptions of liability to the recipient, for
	385	any liability that these contractual assumptions directly impose on
	386	those licensors and authors.
	387
	388	All other non-permissive additional terms are considered "further
	389	restrictions" within the meaning of section 10. If the Program as you
	390	received it, or any part of it, contains a notice stating that it is
	391	governed by this License along with a term that is a further
	392	restriction, you may remove that term. If a license document contains
	393	a further restriction but permits relicensing or conveying under this
	394	License, you may add to a covered work material governed by the terms
	395	of that license document, provided that the further restriction does
	396	not survive such relicensing or conveying.
	397
	398	If you add terms to a covered work in accord with this section, you
	399	must place, in the relevant source files, a statement of the
	400	additional terms that apply to those files, or a notice indicating
	401	where to find the applicable terms.
	402
	403	Additional terms, permissive or non-permissive, may be stated in the
	404	form of a separately written license, or stated as exceptions;
	405	the above requirements apply either way.
	406
	407	8. Termination.
	408
	409	You may not propagate or modify a covered work except as expressly
	410	provided under this License. Any attempt otherwise to propagate or
	411	modify it is void, and will automatically terminate your rights under
	412	this License (including any patent licenses granted under the third
	413	paragraph of section 11).
	414
	415	However, if you cease all violation of this License, then your
	416	license from a particular copyright holder is reinstated (a)
	417	provisionally, unless and until the copyright holder explicitly and
	418	finally terminates your license, and (b) permanently, if the copyright
	419	holder fails to notify you of the violation by some reasonable means
	420	prior to 60 days after the cessation.
	421
	422	Moreover, your license from a particular copyright holder is
	423	reinstated permanently if the copyright holder notifies you of the
	424	violation by some reasonable means, this is the first time you have
	425	received notice of violation of this License (for any work) from that
	426	copyright holder, and you cure the violation prior to 30 days after
	427	your receipt of the notice.
	428
	429	Termination of your rights under this section does not terminate the
	430	licenses of parties who have received copies or rights from you under
	431	this License. If your rights have been terminated and not permanently
	432	reinstated, you do not qualify to receive new licenses for the same
	433	material under section 10.
	434
	435	9. Acceptance Not Required for Having Copies.
	436
	437	You are not required to accept this License in order to receive or
	438	run a copy of the Program. Ancillary propagation of a covered work
	439	occurring solely as a consequence of using peer-to-peer transmission
	440	to receive a copy likewise does not require acceptance. However,
	441	nothing other than this License grants you permission to propagate or
	442	modify any covered work. These actions infringe copyright if you do
	443	not accept this License. Therefore, by modifying or propagating a
	444	covered work, you indicate your acceptance of this License to do so.
	445
	446	10. Automatic Licensing of Downstream Recipients.
	447
	448	Each time you convey a covered work, the recipient automatically
	449	receives a license from the original licensors, to run, modify and
	450	propagate that work, subject to this License. You are not responsible
	451	for enforcing compliance by third parties with this License.
	452
	453	An "entity transaction" is a transaction transferring control of an
	454	organization, or substantially all assets of one, or subdividing an
	455	organization, or merging organizations. If propagation of a covered
	456	work results from an entity transaction, each party to that
	457	transaction who receives a copy of the work also receives whatever
	458	licenses to the work the party's predecessor in interest had or could
	459	give under the previous paragraph, plus a right to possession of the
	460	Corresponding Source of the work from the predecessor in interest, if
	461	the predecessor has it or can get it with reasonable efforts.
	462
	463	You may not impose any further restrictions on the exercise of the
	464	rights granted or affirmed under this License. For example, you may
	465	not impose a license fee, royalty, or other charge for exercise of
	466	rights granted under this License, and you may not initiate litigation
	467	(including a cross-claim or counterclaim in a lawsuit) alleging that
	468	any patent claim is infringed by making, using, selling, offering for
	469	sale, or importing the Program or any portion of it.
	470
	471	11. Patents.
	472
	473	A "contributor" is a copyright holder who authorizes use under this
	474	License of the Program or a work on which the Program is based. The
	475	work thus licensed is called the contributor's "contributor version".
	476
	477	A contributor's "essential patent claims" are all patent claims
	478	owned or controlled by the contributor, whether already acquired or
	479	hereafter acquired, that would be infringed by some manner, permitted
	480	by this License, of making, using, or selling its contributor version,
	481	but do not include claims that would be infringed only as a
	482	consequence of further modification of the contributor version. For
	483	purposes of this definition, "control" includes the right to grant
	484	patent sublicenses in a manner consistent with the requirements of
	485	this License.
	486
	487	Each contributor grants you a non-exclusive, worldwide, royalty-free
	488	patent license under the contributor's essential patent claims, to
	489	make, use, sell, offer for sale, import and otherwise run, modify and
	490	propagate the contents of its contributor version.
	491
	492	In the following three paragraphs, a "patent license" is any express
	493	agreement or commitment, however denominated, not to enforce a patent
	494	(such as an express permission to practice a patent or covenant not to
	495	sue for patent infringement). To "grant" such a patent license to a
	496	party means to make such an agreement or commitment not to enforce a
	497	patent against the party.
	498
	499	If you convey a covered work, knowingly relying on a patent license,
	500	and the Corresponding Source of the work is not available for anyone
	501	to copy, free of charge and under the terms of this License, through a
	502	publicly available network server or other readily accessible means,
	503	then you must either (1) cause the Corresponding Source to be so
	504	available, or (2) arrange to deprive yourself of the benefit of the
	505	patent license for this particular work, or (3) arrange, in a manner
	506	consistent with the requirements of this License, to extend the patent
	507	license to downstream recipients. "Knowingly relying" means you have
	508	actual knowledge that, but for the patent license, your conveying the
	509	covered work in a country, or your recipient's use of the covered work
	510	in a country, would infringe one or more identifiable patents in that
	511	country that you have reason to believe are valid.
	512
	513	If, pursuant to or in connection with a single transaction or
	514	arrangement, you convey, or propagate by procuring conveyance of, a
	515	covered work, and grant a patent license to some of the parties
	516	receiving the covered work authorizing them to use, propagate, modify
	517	or convey a specific copy of the covered work, then the patent license
	518	you grant is automatically extended to all recipients of the covered
	519	work and works based on it.
	520
	521	A patent license is "discriminatory" if it does not include within
	522	the scope of its coverage, prohibits the exercise of, or is
	523	conditioned on the non-exercise of one or more of the rights that are
	524	specifically granted under this License. You may not convey a covered
	525	work if you are a party to an arrangement with a third party that is
	526	in the business of distributing software, under which you make payment
	527	to the third party based on the extent of your activity of conveying
	528	the work, and under which the third party grants, to any of the
	529	parties who would receive the covered work from you, a discriminatory
	530	patent license (a) in connection with copies of the covered work
	531	conveyed by you (or copies made from those copies), or (b) primarily
	532	for and in connection with specific products or compilations that
	533	contain the covered work, unless you entered into that arrangement,
	534	or that patent license was granted, prior to 28 March 2007.
	535
	536	Nothing in this License shall be construed as excluding or limiting
	537	any implied license or other defenses to infringement that may
	538	otherwise be available to you under applicable patent law.
	539
	540	12. No Surrender of Others' Freedom.
	541
	542	If conditions are imposed on you (whether by court order, agreement or
	543	otherwise) that contradict the conditions of this License, they do not
	544	excuse you from the conditions of this License. If you cannot convey a
	545	covered work so as to satisfy simultaneously your obligations under this
	546	License and any other pertinent obligations, then as a consequence you may
	547	not convey it at all. For example, if you agree to terms that obligate you
	548	to collect a royalty for further conveying from those to whom you convey
	549	the Program, the only way you could satisfy both those terms and this
	550	License would be to refrain entirely from conveying the Program.
	551
	552	13. Use with the GNU Affero General Public License.
	553
	554	Notwithstanding any other provision of this License, you have
	555	permission to link or combine any covered work with a work licensed
	556	under version 3 of the GNU Affero General Public License into a single
	557	combined work, and to convey the resulting work. The terms of this
	558	License will continue to apply to the part which is the covered work,
	559	but the special requirements of the GNU Affero General Public License,
	560	section 13, concerning interaction through a network will apply to the
	561	combination as such.
	562
	563	14. Revised Versions of this License.
	564
	565	The Free Software Foundation may publish revised and/or new versions of
	566	the GNU General Public License from time to time. Such new versions will
	567	be similar in spirit to the present version, but may differ in detail to
	568	address new problems or concerns.
	569
	570	Each version is given a distinguishing version number. If the
	571	Program specifies that a certain numbered version of the GNU General
	572	Public License "or any later version" applies to it, you have the
	573	option of following the terms and conditions either of that numbered
	574	version or of any later version published by the Free Software
	575	Foundation. If the Program does not specify a version number of the
	576	GNU General Public License, you may choose any version ever published
	577	by the Free Software Foundation.
	578
	579	If the Program specifies that a proxy can decide which future
	580	versions of the GNU General Public License can be used, that proxy's
	581	public statement of acceptance of a version permanently authorizes you
	582	to choose that version for the Program.
	583
	584	Later license versions may give you additional or different
	585	permissions. However, no additional obligations are imposed on any
	586	author or copyright holder as a result of your choosing to follow a
	587	later version.
	588
	589	15. Disclaimer of Warranty.
	590
	591	THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY
	592	APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT
	593	HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY
	594	OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO,
	595	THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
	596	PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM
	597	IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF
	598	ALL NECESSARY SERVICING, REPAIR OR CORRECTION.
	599
	600	16. Limitation of Liability.
	601
	602	IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
	603	WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS
	604	THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY
	605	GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE
	606	USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF
	607	DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD
	608	PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS),
	609	EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF
	610	SUCH DAMAGES.
	611
	612	17. Interpretation of Sections 15 and 16.
	613
	614	If the disclaimer of warranty and limitation of liability provided
	615	above cannot be given local legal effect according to their terms,
	616	reviewing courts shall apply local law that most closely approximates
	617	an absolute waiver of all civil liability in connection with the
	618	Program, unless a warranty or assumption of liability accompanies a
	619	copy of the Program in return for a fee.
	620
	621	END OF TERMS AND CONDITIONS
	622
	623	How to Apply These Terms to Your New Programs
	624
	625	If you develop a new program, and you want it to be of the greatest
	626	possible use to the public, the best way to achieve this is to make it
	627	free software which everyone can redistribute and change under these terms.
	628
	629	To do so, attach the following notices to the program. It is safest
	630	to attach them to the start of each source file to most effectively
	631	state the exclusion of warranty; and each file should have at least
	632	the "copyright" line and a pointer to where the full notice is found.
	633
	634	<one line to give the program's name and a brief idea of what it does.>
	635	Copyright (C) <year> <name of author>
	636
	637	This program is free software: you can redistribute it and/or modify
	638	it under the terms of the GNU General Public License as published by
	639	the Free Software Foundation, either version 3 of the License, or
	640	(at your option) any later version.
	641
	642	This program is distributed in the hope that it will be useful,
	643	but WITHOUT ANY WARRANTY; without even the implied warranty of
	644	MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
	645	GNU General Public License for more details.
	646
	647	You should have received a copy of the GNU General Public License
	648	along with this program. If not, see <https://www.gnu.org/licenses/>.
	649
	650	Also add information on how to contact you by electronic and paper mail.
	651
	652	If the program does terminal interaction, make it output a short
	653	notice like this when it starts in an interactive mode:
	654
	655	<program> Copyright (C) <year> <name of author>
	656	This program comes with ABSOLUTELY NO WARRANTY; for details type `show w'.
	657	This is free software, and you are welcome to redistribute it
	658	under certain conditions; type `show c' for details.
	659
	660	The hypothetical commands `show w' and `show c' should show the appropriate
	661	parts of the General Public License. Of course, your program's commands
	662	might be different; for a GUI interface, you would use an "about box".
	663
	664	You should also get your employer (if you work as a programmer) or school,
	665	if any, to sign a "copyright disclaimer" for the program, if necessary.
	666	For more information on this, and how to apply and follow the GNU GPL, see
	667	<https://www.gnu.org/licenses/>.
	668
	669	The GNU General Public License does not permit incorporating your program
	670	into proprietary programs. If your program is a subroutine library, you
	671	may consider it more useful to permit linking proprietary applications with
	672	the library. If this is what you want to do, use the GNU Lesser General
	673	Public License instead of this License. But first, please read
	674	<https://www.gnu.org/licenses/why-not-lgpl.html>.

README.md added +184

@@ -0,0 +1,184 @@
	1	# control-coverage
	2
	3	[![License: GPL v3](https://img.shields.io/badge/License-GPLv3-blue.svg)](LICENSE)
	4	[![Python](https://img.shields.io/badge/python-3.10%2B-blue.svg)]()
	5
	6	Control-first coverage and blind-spot analysis over an evidence corpus.
	7
	8	The rest of the Audit Labs toolchain is evidence-first: [audit-tools](https://github.com/audit-labs/audit-tools)
	9	collects raw signals, [audit-report](https://github.com/audit-labs/audit-report)
	10	maps each finding onto the controls it touches, and [evidence-seal](https://github.com/audit-labs/evidence-seal)
	11	proves the package is authentic. That answers *"what did I collect, and what does it
	12	map to?"* — but it can never tell you what you are not looking at, because it has
	13	no list of everything a framework requires.
	14
	15	`control-coverage` supplies that missing list — the denominator. It starts from
	16	the complete catalog of a framework's controls and scores your evidence against it,
	17	so it can report two numbers nothing else in the pipeline can:
	18
	19	- Coverage % — of everything the framework requires, how much the evidence corpus
	20	addresses at all.
	21	- Blind spots — the in-scope controls that no finding touches. These are the
	22	gaps an auditor finds for you if you don't find them first.
	23
	24	It also produces a Statement of Applicability — the ISO 27001 artifact that lists
	25	every Annex A control, whether it applies, and why — derived from your evidence
	26	instead of hand-maintained.
	27
	28	> Like every Audit Labs tool, this produces evidence, not a verdict. An unaddressed
	29	> control is a gap in evidence, which may reflect a real gap in controls or simply
	30	> a signal not yet collected. The final judgment belongs to the organization and its
	31	> auditor.
	32
	33	## Install
	34
	35	```bash
	36	git clone https://github.com/audit-labs/control-coverage
	37	cd control-coverage
	38	python -m venv .venv && source .venv/bin/activate
	39	pip install -e .
	40	```
	41
	42	Pure standard library plus PyYAML — no other dependencies.
	43
	44	## Usage
	45
	46	The input is one or more JSON reports from `audit-report` (its `--format json`
	47	output). A corpus is typically one report per platform and date — AWS, GitHub,
	48	GitLab — which `control-coverage` folds into a single per-framework picture.
	49
	50	```bash
	51	# Coverage across every framework the corpus cites, Markdown to stdout
	52	control-coverage aws.json github.json
	53
	54	# Just the blind spots — the controls nothing evidences yet
	55	control-coverage aws.json github.json --framework SOC2 --blind-spots
	56
	57	# A whole directory of reports, all formats into ./out/
	58	control-coverage ./reports/ --format md,html,json,soa --out out/
	59
	60	# Gate CI: exit non-zero if any framework's coverage is under 60%
	61	control-coverage ./reports/ --fail-under 60
	62	```
	63
	64	### Trend — how coverage moved
	65
	66	Point `--baseline` at an earlier corpus (a file or a directory) to see what changed:
	67	controls that improved, regressed, and — the two that move the coverage number —
	68	were gained (a blind spot became addressed) or lost (an addressed control became
	69	a blind spot).
	70
	71	```bash
	72	control-coverage ./reports/2026-02/ --baseline ./reports/2026-01/ --framework SOC2
	73
	74	# Gate CI: fail the build if any control regressed or lost coverage
	75	control-coverage ./reports/2026-02/ --baseline ./reports/2026-01/ --fail-on-regression
	76	```
	77
	78	Trend mode outputs Markdown, HTML, or JSON (`--format md,html,json`).
	79
	80	### Crosswalk — evidence leverage and the minimal set
	81
	82	One check is rarely worth one control: enforced 2FA is evidence for SOC 2 CC6.1,
	83	ISO A.5.17, and NIST IA-2 at once. `--crosswalk` shows that leverage per check and
	84	computes the minimal evidence set — the fewest checks that still touch every
	85	addressed control, which is what you want when scoping a walkthrough or a sample.
	86
	87	```bash
	88	control-coverage ./reports/ --framework SOC2,ISO,NIST --crosswalk
	89	```
	90
	91	Crosswalk mode outputs Markdown, HTML, or JSON (`--format md,html,json`).
	92
	93	### Scope and the Statement of Applicability
	94
	95	Not every control applies to every organization. A scope file records which
	96	controls are excluded and — required, never optional — why:
	97
	98	```yaml
	99	# soa.yaml
	100	subject: Acme Production
	101	frameworks: [SOC2, ISO]
	102	exclusions:
	103	- control: ISO:A.7.1
	104	reason: "Fully cloud-hosted; no physical premises are in scope for the ISMS."
	105	- control: ISO:A.5.7
	106	reason: "No formal threat-intelligence program; risk accepted by the CISO for 2026."
	107	owners:
	108	SOC2:CC6.1: platform-team
	109	```
	110
	111	```bash
	112	# Coverage over in-scope controls, plus a ready-to-file SoA
	113	control-coverage ./reports/ --scope soa.yaml --format md,soa --out out/
	114	```
	115
	116	Excluded controls are recorded with their justification rather than counted as gaps.
	117	An exclusion with no reason is rejected — an unjustified exclusion is the single most
	118	common SoA audit finding.
	119
	120	## Assurance states
	121
	122	Every in-scope control lands in exactly one state:
	123
	124	\| State \| Meaning \|
	125	\| --- \| --- \|
	126	\| supported \| At least one mapped finding passes, and none fail. \|
	127	\| failing \| At least one mapped finding fails. The worst observation wins. \|
	128	\| asserted \| Findings map here, but their data was absent — evidence attempted, not obtained. \|
	129	\| unaddressed \| No finding maps here at all. The blind spot. \|
	130	\| out of scope \| Excluded by the scope file, with a recorded justification. \|
	131
	132	`coverage %` is the share of in-scope controls in any of the first three states;
	133	`assured %` is the share that are `supported`.
	134
	135	## Bundled catalogs
	136
	137	\| Framework \| Code \| Catalog \|
	138	\| --- \| --- \| --- \|
	139	\| SOC 2 (Trust Services Criteria) \| `SOC2` \| Complete — all five categories (Common Criteria + Availability, Confidentiality, Processing Integrity, Privacy), 61 controls \|
	140	\| ISO/IEC 27001:2022 Annex A \| `ISO` \| Complete — all 93 controls \|
	141	\| NIST SP 800-53 Rev. 5 \| `NIST` \| Moderate baseline — 177 base controls across 18 families \|
	142
	143	Most SOC 2 reports scope only some categories (Security is near-universal; Privacy and
	144	Processing Integrity often are not). Use `exclude_families` in the scope file to drop a
	145	whole category — or an ISO theme, or a NIST family — from the denominator in one line:
	146
	147	```yaml
	148	exclude_families:
	149	- {framework: SOC2, family: Privacy, reason: "Privacy category not in the SOC 2 audit scope."}
	150	- {framework: SOC2, family: Processing Integrity, reason: "PI category not in the SOC 2 audit scope."}
	151	```
	152
	153	Control codes are written `FRAMEWORK:ID` (`SOC2:CC6.1`, `ISO:A.5.17`), matching the
	154	codes `audit-report` rulesets already cite. A partial catalog is reported honestly as
	155	coverage of the shipped subset, never as the whole standard.
	156
	157	If the corpus cites a code whose framework is loaded but the catalog does not define
	158	it — a typo or a renamed control — it is surfaced as an unmatched control code
	159	rather than silently ignored.
	160
	161	## How it fits the pipeline
	162
	163	```
	164	audit-tools ──► CSV package ──► evidence-seal (seal + verify)
	165	│
	166	▼
	167	audit-report ──► per-package report (--format json)
	168	│
	169	▼ one or more reports = a corpus
	170	control-coverage ──► coverage %, blind spots, SoA,
	171	trend over time, evidence crosswalk
	172	```
	173
	174	## Development
	175
	176	```bash
	177	pip install -e ".[dev]"
	178	pytest
	179	ruff check .
	180	```
	181
	182	## License
	183
	184	GPL-3.0-or-later. See [LICENSE](LICENSE).

conftest.py added +11

@@ -0,0 +1,11 @@
	1	"""Ensure the project root is importable and tests run from it.
	2
	3	Tests reference the bundled catalogs by the relative path
	4	``control_coverage/catalogs``, so pytest must be invoked from the project root.
	5	This file's location pins that root for import resolution.
	6	"""
	7
	8	import sys
	9	from pathlib import Path
	10
	11	sys.path.insert(0, str(Path(__file__).parent))

control_coverage/__init__.py added +3

@@ -0,0 +1,3 @@
	1	"""control-coverage — control-first coverage and blind-spot analysis over an evidence corpus."""
	2
	3	__version__ = "0.1.0"

control_coverage/__main__.py added +6

@@ -0,0 +1,6 @@
	1	"""Enable ``python -m control_coverage``."""
	2
	3	from .cli import main
	4
	5	if __name__ == "__main__":
	6	raise SystemExit(main())

control_coverage/catalog.py added +116

@@ -0,0 +1,116 @@
	1	"""Framework catalogs — the complete list of controls a framework defines.
	2
	3	This is the piece the rest of the audit-labs ecosystem does not have. Tools like
	4	audit-report are evidence-first: they start from what you collected and map each
	5	finding onto whatever controls it touches. That can never tell you what you are
	6	not looking at, because it has no list of everything a framework requires.
	7
	8	A catalog is that list — the denominator. Loading ``soc2`` gives every Trust
	9	Services Criterion; loading ``iso`` gives all 93 Annex A controls. Coverage is
	10	then simply: of these, how many does the evidence corpus actually address?
	11
	12	Control codes are written ``FRAMEWORK:ID`` (for example ``SOC2:CC6.1``,
	13	``ISO:A.5.17``), matching the codes audit-report rulesets cite.
	14	"""
	15
	16	from __future__ import annotations
	17
	18	from dataclasses import dataclass
	19	from pathlib import Path
	20
	21	import yaml
	22
	23	_CATALOG_DIR = Path(__file__).resolve().parent / "catalogs"
	24
	25	# User-facing framework name -> catalog file stem. Aliases keep the CLI forgiving.
	26	_ALIASES = {
	27	"soc2": "soc2",
	28	"soc 2": "soc2",
	29	"iso": "iso27001",
	30	"iso27001": "iso27001",
	31	"iso 27001": "iso27001",
	32	"nist": "nist80053",
	33	"nist80053": "nist80053",
	34	"800-53": "nist80053",
	35	}
	36
	37
	38	@dataclass(frozen=True)
	39	class Control:
	40	"""One control in a framework catalog."""
	41
	42	framework: str
	43	id: str
	44	title: str
	45	family: str = ""
	46
	47	@property
	48	def code(self) -> str:
	49	"""The full ``FRAMEWORK:ID`` code used to join against evidence."""
	50	return f"{self.framework}:{self.id}"
	51
	52
	53	@dataclass
	54	class Catalog:
	55	"""A framework's complete (or explicitly partial) set of controls."""
	56
	57	framework: str
	58	name: str
	59	version: str
	60	coverage: str # "complete" or "partial"
	61	source: str
	62	controls: list[Control]
	63
	64	@property
	65	def complete(self) -> bool:
	66	return self.coverage == "complete"
	67
	68	def codes(self) -> set[str]:
	69	return {c.code for c in self.controls}
	70
	71
	72	def available() -> list[str]:
	73	"""Framework short codes with a bundled catalog (e.g. ``["ISO", "NIST", "SOC2"]``)."""
	74	return sorted(load(p.stem).framework for p in _CATALOG_DIR.glob("*.yaml"))
	75
	76
	77	def _resolve(name: str) -> Path:
	78	stem = _ALIASES.get(name.strip().lower(), name.strip().lower())
	79	path = _CATALOG_DIR / f"{stem}.yaml"
	80	if not path.exists():
	81	known = ", ".join(sorted(p.stem for p in _CATALOG_DIR.glob("*.yaml")))
	82	raise ValueError(f"unknown framework '{name}'. Bundled catalogs: {known}")
	83	return path
	84
	85
	86	def load(name: str) -> Catalog:
	87	"""Load a bundled catalog by framework name, short code, or alias."""
	88	path = _resolve(name)
	89	raw = yaml.safe_load(path.read_text(encoding="utf-8"))
	90	framework = raw["framework"]
	91	controls = [
	92	Control(
	93	framework=framework,
	94	id=str(c["id"]),
	95	title=str(c["title"]),
	96	family=str(c.get("family", "")),
	97	)
	98	for c in raw.get("controls", [])
	99	]
	100	return Catalog(
	101	framework=framework,
	102	name=raw.get("name", framework),
	103	version=str(raw.get("version", "")),
	104	coverage=raw.get("coverage", "partial"),
	105	source=raw.get("source", ""),
	106	controls=controls,
	107	)
	108
	109
	110	def load_frameworks(names: list[str]) -> list[Catalog]:
	111	"""Load several catalogs, de-duplicated by framework, in a stable order."""
	112	seen: dict[str, Catalog] = {}
	113	for name in names:
	114	cat = load(name)
	115	seen[cat.framework] = cat
	116	return [seen[k] for k in sorted(seen)]

control_coverage/catalogs/iso27001.yaml added +107

@@ -0,0 +1,107 @@
	1	# ISO/IEC 27001:2022 — Annex A (all 93 controls, four themes).
	2	#
	3	# This is exactly the list a Statement of Applicability enumerates. A control's
	4	# full code is "ISO:<id>", matching the codes audit-report rulesets cite.
	5	framework: ISO
	6	name: ISO/IEC 27001:2022 Annex A
	7	version: "2022"
	8	coverage: complete
	9	source: ISO/IEC 27001:2022 Annex A
	10	controls:
	11	# A.5 — Organizational controls
	12	- {id: A.5.1, family: Organizational, title: "Policies for information security."}
	13	- {id: A.5.2, family: Organizational, title: "Information security roles and responsibilities."}
	14	- {id: A.5.3, family: Organizational, title: "Segregation of duties."}
	15	- {id: A.5.4, family: Organizational, title: "Management responsibilities."}
	16	- {id: A.5.5, family: Organizational, title: "Contact with authorities."}
	17	- {id: A.5.6, family: Organizational, title: "Contact with special interest groups."}
	18	- {id: A.5.7, family: Organizational, title: "Threat intelligence."}
	19	- {id: A.5.8, family: Organizational, title: "Information security in project management."}
	20	- {id: A.5.9, family: Organizational, title: "Inventory of information and other associated assets."}
	21	- {id: A.5.10, family: Organizational, title: "Acceptable use of information and other associated assets."}
	22	- {id: A.5.11, family: Organizational, title: "Return of assets."}
	23	- {id: A.5.12, family: Organizational, title: "Classification of information."}
	24	- {id: A.5.13, family: Organizational, title: "Labelling of information."}
	25	- {id: A.5.14, family: Organizational, title: "Information transfer."}
	26	- {id: A.5.15, family: Organizational, title: "Access control."}
	27	- {id: A.5.16, family: Organizational, title: "Identity management."}
	28	- {id: A.5.17, family: Organizational, title: "Authentication information."}
	29	- {id: A.5.18, family: Organizational, title: "Access rights."}
	30	- {id: A.5.19, family: Organizational, title: "Information security in supplier relationships."}
	31	- {id: A.5.20, family: Organizational, title: "Addressing information security within supplier agreements."}
	32	- {id: A.5.21, family: Organizational, title: "Managing information security in the ICT supply chain."}
	33	- {id: A.5.22, family: Organizational, title: "Monitoring, review and change management of supplier services."}
	34	- {id: A.5.23, family: Organizational, title: "Information security for use of cloud services."}
	35	- {id: A.5.24, family: Organizational, title: "Information security incident management planning and preparation."}
	36	- {id: A.5.25, family: Organizational, title: "Assessment and decision on information security events."}
	37	- {id: A.5.26, family: Organizational, title: "Response to information security incidents."}
	38	- {id: A.5.27, family: Organizational, title: "Learning from information security incidents."}
	39	- {id: A.5.28, family: Organizational, title: "Collection of evidence."}
	40	- {id: A.5.29, family: Organizational, title: "Information security during disruption."}
	41	- {id: A.5.30, family: Organizational, title: "ICT readiness for business continuity."}
	42	- {id: A.5.31, family: Organizational, title: "Legal, statutory, regulatory and contractual requirements."}
	43	- {id: A.5.32, family: Organizational, title: "Intellectual property rights."}
	44	- {id: A.5.33, family: Organizational, title: "Protection of records."}
	45	- {id: A.5.34, family: Organizational, title: "Privacy and protection of personally identifiable information (PII)."}
	46	- {id: A.5.35, family: Organizational, title: "Independent review of information security."}
	47	- {id: A.5.36, family: Organizational, title: "Compliance with policies, rules and standards for information security."}
	48	- {id: A.5.37, family: Organizational, title: "Documented operating procedures."}
	49	# A.6 — People controls
	50	- {id: A.6.1, family: People, title: "Screening."}
	51	- {id: A.6.2, family: People, title: "Terms and conditions of employment."}
	52	- {id: A.6.3, family: People, title: "Information security awareness, education and training."}
	53	- {id: A.6.4, family: People, title: "Disciplinary process."}
	54	- {id: A.6.5, family: People, title: "Responsibilities after termination or change of employment."}
	55	- {id: A.6.6, family: People, title: "Confidentiality or non-disclosure agreements."}
	56	- {id: A.6.7, family: People, title: "Remote working."}
	57	- {id: A.6.8, family: People, title: "Information security event reporting."}
	58	# A.7 — Physical controls
	59	- {id: A.7.1, family: Physical, title: "Physical security perimeters."}
	60	- {id: A.7.2, family: Physical, title: "Physical entry."}
	61	- {id: A.7.3, family: Physical, title: "Securing offices, rooms and facilities."}
	62	- {id: A.7.4, family: Physical, title: "Physical security monitoring."}
	63	- {id: A.7.5, family: Physical, title: "Protecting against physical and environmental threats."}
	64	- {id: A.7.6, family: Physical, title: "Working in secure areas."}
	65	- {id: A.7.7, family: Physical, title: "Clear desk and clear screen."}
	66	- {id: A.7.8, family: Physical, title: "Equipment siting and protection."}
	67	- {id: A.7.9, family: Physical, title: "Security of assets off-premises."}
	68	- {id: A.7.10, family: Physical, title: "Storage media."}
	69	- {id: A.7.11, family: Physical, title: "Supporting utilities."}
	70	- {id: A.7.12, family: Physical, title: "Cabling security."}
	71	- {id: A.7.13, family: Physical, title: "Equipment maintenance."}
	72	- {id: A.7.14, family: Physical, title: "Secure disposal or re-use of equipment."}
	73	# A.8 — Technological controls
	74	- {id: A.8.1, family: Technological, title: "User endpoint devices."}
	75	- {id: A.8.2, family: Technological, title: "Privileged access rights."}
	76	- {id: A.8.3, family: Technological, title: "Information access restriction."}
	77	- {id: A.8.4, family: Technological, title: "Access to source code."}
	78	- {id: A.8.5, family: Technological, title: "Secure authentication."}
	79	- {id: A.8.6, family: Technological, title: "Capacity management."}
	80	- {id: A.8.7, family: Technological, title: "Protection against malware."}
	81	- {id: A.8.8, family: Technological, title: "Management of technical vulnerabilities."}
	82	- {id: A.8.9, family: Technological, title: "Configuration management."}
	83	- {id: A.8.10, family: Technological, title: "Information deletion."}
	84	- {id: A.8.11, family: Technological, title: "Data masking."}
	85	- {id: A.8.12, family: Technological, title: "Data leakage prevention."}
	86	- {id: A.8.13, family: Technological, title: "Information backup."}
	87	- {id: A.8.14, family: Technological, title: "Redundancy of information processing facilities."}
	88	- {id: A.8.15, family: Technological, title: "Logging."}
	89	- {id: A.8.16, family: Technological, title: "Monitoring activities."}
	90	- {id: A.8.17, family: Technological, title: "Clock synchronization."}
	91	- {id: A.8.18, family: Technological, title: "Use of privileged utility programs."}
	92	- {id: A.8.19, family: Technological, title: "Installation of software on operational systems."}
	93	- {id: A.8.20, family: Technological, title: "Networks security."}
	94	- {id: A.8.21, family: Technological, title: "Security of network services."}
	95	- {id: A.8.22, family: Technological, title: "Segregation of networks."}
	96	- {id: A.8.23, family: Technological, title: "Web filtering."}
	97	- {id: A.8.24, family: Technological, title: "Use of cryptography."}
	98	- {id: A.8.25, family: Technological, title: "Secure development life cycle."}
	99	- {id: A.8.26, family: Technological, title: "Application security requirements."}
	100	- {id: A.8.27, family: Technological, title: "Secure system architecture and engineering principles."}
	101	- {id: A.8.28, family: Technological, title: "Secure coding."}
	102	- {id: A.8.29, family: Technological, title: "Security testing in development and acceptance."}
	103	- {id: A.8.30, family: Technological, title: "Outsourced development."}
	104	- {id: A.8.31, family: Technological, title: "Separation of development, test and production environments."}
	105	- {id: A.8.32, family: Technological, title: "Change management."}
	106	- {id: A.8.33, family: Technological, title: "Test information."}
	107	- {id: A.8.34, family: Technological, title: "Protection of information systems during audit testing."}

control_coverage/catalogs/nist80053.yaml added +210

@@ -0,0 +1,210 @@
	1	# NIST SP 800-53 Rev. 5 — Moderate baseline (base controls).
	2	#
	3	# The base controls selected in the SP 800-53B moderate impact baseline, across
	4	# all 18 baseline-applicable families. Control enhancements (e.g. AC-2(1)) are not
	5	# enumerated — coverage is measured at the base-control level. The program
	6	# management (PM) family is org-wide and not baseline-allocated; the privacy (PT)
	7	# family is selected via the separate privacy baseline. A control's full code is
	8	# "NIST:<id>".
	9	framework: NIST
	10	name: NIST SP 800-53 Rev. 5 (Moderate baseline)
	11	version: "Rev. 5"
	12	baseline: Moderate
	13	coverage: complete
	14	source: NIST SP 800-53B Moderate baseline (base controls)
	15	controls:
	16	# AC — Access Control
	17	- {id: AC-1, family: "Access Control", title: "Policy and Procedures"}
	18	- {id: AC-2, family: "Access Control", title: "Account Management"}
	19	- {id: AC-3, family: "Access Control", title: "Access Enforcement"}
	20	- {id: AC-4, family: "Access Control", title: "Information Flow Enforcement"}
	21	- {id: AC-5, family: "Access Control", title: "Separation of Duties"}
	22	- {id: AC-6, family: "Access Control", title: "Least Privilege"}
	23	- {id: AC-7, family: "Access Control", title: "Unsuccessful Logon Attempts"}
	24	- {id: AC-8, family: "Access Control", title: "System Use Notification"}
	25	- {id: AC-11, family: "Access Control", title: "Device Lock"}
	26	- {id: AC-12, family: "Access Control", title: "Session Termination"}
	27	- {id: AC-14, family: "Access Control", title: "Permitted Actions Without Identification or Authentication"}
	28	- {id: AC-17, family: "Access Control", title: "Remote Access"}
	29	- {id: AC-18, family: "Access Control", title: "Wireless Access"}
	30	- {id: AC-19, family: "Access Control", title: "Access Control for Mobile Devices"}
	31	- {id: AC-20, family: "Access Control", title: "Use of External Systems"}
	32	- {id: AC-21, family: "Access Control", title: "Information Sharing"}
	33	- {id: AC-22, family: "Access Control", title: "Publicly Accessible Content"}
	34	# AT — Awareness and Training
	35	- {id: AT-1, family: "Awareness and Training", title: "Policy and Procedures"}
	36	- {id: AT-2, family: "Awareness and Training", title: "Literacy Training and Awareness"}
	37	- {id: AT-3, family: "Awareness and Training", title: "Role-Based Training"}
	38	- {id: AT-4, family: "Awareness and Training", title: "Training Records"}
	39	# AU — Audit and Accountability
	40	- {id: AU-1, family: "Audit and Accountability", title: "Policy and Procedures"}
	41	- {id: AU-2, family: "Audit and Accountability", title: "Event Logging"}
	42	- {id: AU-3, family: "Audit and Accountability", title: "Content of Audit Records"}
	43	- {id: AU-4, family: "Audit and Accountability", title: "Audit Log Storage Capacity"}
	44	- {id: AU-5, family: "Audit and Accountability", title: "Response to Audit Logging Process Failures"}
	45	- {id: AU-6, family: "Audit and Accountability", title: "Audit Record Review, Analysis, and Reporting"}
	46	- {id: AU-7, family: "Audit and Accountability", title: "Audit Record Reduction and Report Generation"}
	47	- {id: AU-8, family: "Audit and Accountability", title: "Time Stamps"}
	48	- {id: AU-9, family: "Audit and Accountability", title: "Protection of Audit Information"}
	49	- {id: AU-11, family: "Audit and Accountability", title: "Audit Record Retention"}
	50	- {id: AU-12, family: "Audit and Accountability", title: "Audit Record Generation"}
	51	# CA — Assessment, Authorization, and Monitoring
	52	- {id: CA-1, family: "Assessment, Authorization, and Monitoring", title: "Policy and Procedures"}
	53	- {id: CA-2, family: "Assessment, Authorization, and Monitoring", title: "Control Assessments"}
	54	- {id: CA-3, family: "Assessment, Authorization, and Monitoring", title: "Information Exchange"}
	55	- {id: CA-5, family: "Assessment, Authorization, and Monitoring", title: "Plan of Action and Milestones"}
	56	- {id: CA-6, family: "Assessment, Authorization, and Monitoring", title: "Authorization"}
	57	- {id: CA-7, family: "Assessment, Authorization, and Monitoring", title: "Continuous Monitoring"}
	58	- {id: CA-9, family: "Assessment, Authorization, and Monitoring", title: "Internal System Connections"}
	59	# CM — Configuration Management
	60	- {id: CM-1, family: "Configuration Management", title: "Policy and Procedures"}
	61	- {id: CM-2, family: "Configuration Management", title: "Baseline Configuration"}
	62	- {id: CM-3, family: "Configuration Management", title: "Configuration Change Control"}
	63	- {id: CM-4, family: "Configuration Management", title: "Impact Analyses"}
	64	- {id: CM-5, family: "Configuration Management", title: "Access Restrictions for Change"}
	65	- {id: CM-6, family: "Configuration Management", title: "Configuration Settings"}
	66	- {id: CM-7, family: "Configuration Management", title: "Least Functionality"}
	67	- {id: CM-8, family: "Configuration Management", title: "System Component Inventory"}
	68	- {id: CM-9, family: "Configuration Management", title: "Configuration Management Plan"}
	69	- {id: CM-10, family: "Configuration Management", title: "Software Usage Restrictions"}
	70	- {id: CM-11, family: "Configuration Management", title: "User-Installed Software"}
	71	- {id: CM-12, family: "Configuration Management", title: "Information Location"}
	72	# CP — Contingency Planning
	73	- {id: CP-1, family: "Contingency Planning", title: "Policy and Procedures"}
	74	- {id: CP-2, family: "Contingency Planning", title: "Contingency Plan"}
	75	- {id: CP-3, family: "Contingency Planning", title: "Contingency Training"}
	76	- {id: CP-4, family: "Contingency Planning", title: "Contingency Plan Testing"}
	77	- {id: CP-6, family: "Contingency Planning", title: "Alternate Storage Site"}
	78	- {id: CP-7, family: "Contingency Planning", title: "Alternate Processing Site"}
	79	- {id: CP-8, family: "Contingency Planning", title: "Telecommunications Services"}
	80	- {id: CP-9, family: "Contingency Planning", title: "System Backup"}
	81	- {id: CP-10, family: "Contingency Planning", title: "System Recovery and Reconstitution"}
	82	# IA — Identification and Authentication
	83	- {id: IA-1, family: "Identification and Authentication", title: "Policy and Procedures"}
	84	- {id: IA-2, family: "Identification and Authentication", title: "Identification and Authentication (Organizational Users)"}
	85	- {id: IA-3, family: "Identification and Authentication", title: "Device Identification and Authentication"}
	86	- {id: IA-4, family: "Identification and Authentication", title: "Identifier Management"}
	87	- {id: IA-5, family: "Identification and Authentication", title: "Authenticator Management"}
	88	- {id: IA-6, family: "Identification and Authentication", title: "Authentication Feedback"}
	89	- {id: IA-7, family: "Identification and Authentication", title: "Cryptographic Module Authentication"}
	90	- {id: IA-8, family: "Identification and Authentication", title: "Identification and Authentication (Non-Organizational Users)"}
	91	- {id: IA-11, family: "Identification and Authentication", title: "Re-authentication"}
	92	- {id: IA-12, family: "Identification and Authentication", title: "Identity Proofing"}
	93	# IR — Incident Response
	94	- {id: IR-1, family: "Incident Response", title: "Policy and Procedures"}
	95	- {id: IR-2, family: "Incident Response", title: "Incident Response Training"}
	96	- {id: IR-3, family: "Incident Response", title: "Incident Response Testing"}
	97	- {id: IR-4, family: "Incident Response", title: "Incident Handling"}
	98	- {id: IR-5, family: "Incident Response", title: "Incident Monitoring"}
	99	- {id: IR-6, family: "Incident Response", title: "Incident Reporting"}
	100	- {id: IR-7, family: "Incident Response", title: "Incident Response Assistance"}
	101	- {id: IR-8, family: "Incident Response", title: "Incident Response Plan"}
	102	# MA — Maintenance
	103	- {id: MA-1, family: "Maintenance", title: "Policy and Procedures"}
	104	- {id: MA-2, family: "Maintenance", title: "Controlled Maintenance"}
	105	- {id: MA-3, family: "Maintenance", title: "Maintenance Tools"}
	106	- {id: MA-4, family: "Maintenance", title: "Nonlocal Maintenance"}
	107	- {id: MA-5, family: "Maintenance", title: "Maintenance Personnel"}
	108	- {id: MA-6, family: "Maintenance", title: "Timely Maintenance"}
	109	# MP — Media Protection
	110	- {id: MP-1, family: "Media Protection", title: "Policy and Procedures"}
	111	- {id: MP-2, family: "Media Protection", title: "Media Access"}
	112	- {id: MP-3, family: "Media Protection", title: "Media Marking"}
	113	- {id: MP-4, family: "Media Protection", title: "Media Storage"}
	114	- {id: MP-5, family: "Media Protection", title: "Media Transport"}
	115	- {id: MP-6, family: "Media Protection", title: "Media Sanitization"}
	116	- {id: MP-7, family: "Media Protection", title: "Media Use"}
	117	# PE — Physical and Environmental Protection
	118	- {id: PE-1, family: "Physical and Environmental Protection", title: "Policy and Procedures"}
	119	- {id: PE-2, family: "Physical and Environmental Protection", title: "Physical Access Authorizations"}
	120	- {id: PE-3, family: "Physical and Environmental Protection", title: "Physical Access Control"}
	121	- {id: PE-4, family: "Physical and Environmental Protection", title: "Access Control for Transmission"}
	122	- {id: PE-5, family: "Physical and Environmental Protection", title: "Access Control for Output Devices"}
	123	- {id: PE-6, family: "Physical and Environmental Protection", title: "Monitoring Physical Access"}
	124	- {id: PE-8, family: "Physical and Environmental Protection", title: "Visitor Access Records"}
	125	- {id: PE-9, family: "Physical and Environmental Protection", title: "Power Equipment and Cabling"}
	126	- {id: PE-10, family: "Physical and Environmental Protection", title: "Emergency Shutoff"}
	127	- {id: PE-11, family: "Physical and Environmental Protection", title: "Emergency Power"}
	128	- {id: PE-12, family: "Physical and Environmental Protection", title: "Emergency Lighting"}
	129	- {id: PE-13, family: "Physical and Environmental Protection", title: "Fire Protection"}
	130	- {id: PE-14, family: "Physical and Environmental Protection", title: "Environmental Controls"}
	131	- {id: PE-15, family: "Physical and Environmental Protection", title: "Water Damage Protection"}
	132	- {id: PE-16, family: "Physical and Environmental Protection", title: "Delivery and Removal"}
	133	- {id: PE-17, family: "Physical and Environmental Protection", title: "Alternate Work Site"}
	134	# PL — Planning
	135	- {id: PL-1, family: "Planning", title: "Policy and Procedures"}
	136	- {id: PL-2, family: "Planning", title: "System Security and Privacy Plans"}
	137	- {id: PL-4, family: "Planning", title: "Rules of Behavior"}
	138	- {id: PL-8, family: "Planning", title: "Security and Privacy Architectures"}
	139	- {id: PL-10, family: "Planning", title: "Baseline Selection"}
	140	- {id: PL-11, family: "Planning", title: "Baseline Tailoring"}
	141	# PS — Personnel Security
	142	- {id: PS-1, family: "Personnel Security", title: "Policy and Procedures"}
	143	- {id: PS-2, family: "Personnel Security", title: "Position Risk Designation"}
	144	- {id: PS-3, family: "Personnel Security", title: "Personnel Screening"}
	145	- {id: PS-4, family: "Personnel Security", title: "Personnel Termination"}
	146	- {id: PS-5, family: "Personnel Security", title: "Personnel Transfer"}
	147	- {id: PS-6, family: "Personnel Security", title: "Access Agreements"}
	148	- {id: PS-7, family: "Personnel Security", title: "External Personnel Security"}
	149	- {id: PS-8, family: "Personnel Security", title: "Personnel Sanctions"}
	150	- {id: PS-9, family: "Personnel Security", title: "Position Descriptions"}
	151	# RA — Risk Assessment
	152	- {id: RA-1, family: "Risk Assessment", title: "Policy and Procedures"}
	153	- {id: RA-2, family: "Risk Assessment", title: "Security Categorization"}
	154	- {id: RA-3, family: "Risk Assessment", title: "Risk Assessment"}
	155	- {id: RA-5, family: "Risk Assessment", title: "Vulnerability Monitoring and Scanning"}
	156	- {id: RA-7, family: "Risk Assessment", title: "Risk Response"}
	157	# SA — System and Services Acquisition
	158	- {id: SA-1, family: "System and Services Acquisition", title: "Policy and Procedures"}
	159	- {id: SA-2, family: "System and Services Acquisition", title: "Allocation of Resources"}
	160	- {id: SA-3, family: "System and Services Acquisition", title: "System Development Life Cycle"}
	161	- {id: SA-4, family: "System and Services Acquisition", title: "Acquisition Process"}
	162	- {id: SA-5, family: "System and Services Acquisition", title: "System Documentation"}
	163	- {id: SA-8, family: "System and Services Acquisition", title: "Security and Privacy Engineering Principles"}
	164	- {id: SA-9, family: "System and Services Acquisition", title: "External System Services"}
	165	- {id: SA-10, family: "System and Services Acquisition", title: "Developer Configuration Management"}
	166	- {id: SA-11, family: "System and Services Acquisition", title: "Developer Testing and Evaluation"}
	167	- {id: SA-15, family: "System and Services Acquisition", title: "Development Process, Standards, and Tools"}
	168	- {id: SA-22, family: "System and Services Acquisition", title: "Unsupported System Components"}
	169	# SC — System and Communications Protection
	170	- {id: SC-1, family: "System and Communications Protection", title: "Policy and Procedures"}
	171	- {id: SC-2, family: "System and Communications Protection", title: "Separation of System and User Functionality"}
	172	- {id: SC-4, family: "System and Communications Protection", title: "Information in Shared System Resources"}
	173	- {id: SC-5, family: "System and Communications Protection", title: "Denial-of-Service Protection"}
	174	- {id: SC-7, family: "System and Communications Protection", title: "Boundary Protection"}
	175	- {id: SC-8, family: "System and Communications Protection", title: "Transmission Confidentiality and Integrity"}
	176	- {id: SC-10, family: "System and Communications Protection", title: "Network Disconnect"}
	177	- {id: SC-12, family: "System and Communications Protection", title: "Cryptographic Key Establishment and Management"}
	178	- {id: SC-13, family: "System and Communications Protection", title: "Cryptographic Protection"}
	179	- {id: SC-15, family: "System and Communications Protection", title: "Collaborative Computing Devices and Applications"}
	180	- {id: SC-17, family: "System and Communications Protection", title: "Public Key Infrastructure Certificates"}
	181	- {id: SC-18, family: "System and Communications Protection", title: "Mobile Code"}
	182	- {id: SC-20, family: "System and Communications Protection", title: "Secure Name/Address Resolution Service (Authoritative Source)"}
	183	- {id: SC-21, family: "System and Communications Protection", title: "Secure Name/Address Resolution Service (Recursive or Caching Resolver)"}
	184	- {id: SC-22, family: "System and Communications Protection", title: "Architecture and Provisioning for Name/Address Resolution Service"}
	185	- {id: SC-23, family: "System and Communications Protection", title: "Session Authenticity"}
	186	- {id: SC-28, family: "System and Communications Protection", title: "Protection of Information at Rest"}
	187	- {id: SC-39, family: "System and Communications Protection", title: "Process Isolation"}
	188	# SI — System and Information Integrity
	189	- {id: SI-1, family: "System and Information Integrity", title: "Policy and Procedures"}
	190	- {id: SI-2, family: "System and Information Integrity", title: "Flaw Remediation"}
	191	- {id: SI-3, family: "System and Information Integrity", title: "Malicious Code Protection"}
	192	- {id: SI-4, family: "System and Information Integrity", title: "System Monitoring"}
	193	- {id: SI-5, family: "System and Information Integrity", title: "Security Alerts, Advisories, and Directives"}
	194	- {id: SI-7, family: "System and Information Integrity", title: "Software, Firmware, and Information Integrity"}
	195	- {id: SI-8, family: "System and Information Integrity", title: "Spam Protection"}
	196	- {id: SI-10, family: "System and Information Integrity", title: "Information Input Validation"}
	197	- {id: SI-11, family: "System and Information Integrity", title: "Error Handling"}
	198	- {id: SI-12, family: "System and Information Integrity", title: "Information Management and Retention"}
	199	- {id: SI-16, family: "System and Information Integrity", title: "Memory Protection"}
	200	# SR — Supply Chain Risk Management
	201	- {id: SR-1, family: "Supply Chain Risk Management", title: "Policy and Procedures"}
	202	- {id: SR-2, family: "Supply Chain Risk Management", title: "Supply Chain Risk Management Plan"}
	203	- {id: SR-3, family: "Supply Chain Risk Management", title: "Supply Chain Controls and Processes"}
	204	- {id: SR-5, family: "Supply Chain Risk Management", title: "Acquisition Strategies, Tools, and Methods"}
	205	- {id: SR-6, family: "Supply Chain Risk Management", title: "Supplier Assessments and Reviews"}
	206	- {id: SR-8, family: "Supply Chain Risk Management", title: "Notification Agreements"}
	207	- {id: SR-9, family: "Supply Chain Risk Management", title: "Tamper Resistance and Detection"}
	208	- {id: SR-10, family: "Supply Chain Risk Management", title: "Inspection of Systems or Components"}
	209	- {id: SR-11, family: "Supply Chain Risk Management", title: "Component Authenticity"}
	210	- {id: SR-12, family: "Supply Chain Risk Management", title: "Component Disposal"}

control_coverage/catalogs/soc2.yaml added +85

@@ -0,0 +1,85 @@
	1	# SOC 2 — Trust Services Criteria (AICPA, 2017 with 2022 revised points of focus).
	2	#
	3	# The full Common Criteria (the "Security" category every SOC 2 report covers)
	4	# plus the Availability category. A control's full code is "SOC2:<id>", matching
	5	# the codes audit-report rulesets cite.
	6	framework: SOC2
	7	name: SOC 2 (Trust Services Criteria)
	8	version: "2017 (rev. 2022)"
	9	coverage: complete
	10	source: AICPA Trust Services Criteria
	11	controls:
	12	# CC1 — Control Environment
	13	- {id: CC1.1, family: Control Environment, title: "The entity demonstrates a commitment to integrity and ethical values."}
	14	- {id: CC1.2, family: Control Environment, title: "The board of directors demonstrates independence and exercises oversight of internal control."}
	15	- {id: CC1.3, family: Control Environment, title: "Management establishes structures, reporting lines, and appropriate authorities and responsibilities."}
	16	- {id: CC1.4, family: Control Environment, title: "The entity demonstrates a commitment to attract, develop, and retain competent individuals."}
	17	- {id: CC1.5, family: Control Environment, title: "The entity holds individuals accountable for their internal control responsibilities."}
	18	# CC2 — Communication and Information
	19	- {id: CC2.1, family: Communication and Information, title: "The entity obtains or generates relevant, quality information to support internal control."}
	20	- {id: CC2.2, family: Communication and Information, title: "The entity internally communicates information, including objectives and responsibilities for internal control."}
	21	- {id: CC2.3, family: Communication and Information, title: "The entity communicates with external parties about matters affecting internal control."}
	22	# CC3 — Risk Assessment
	23	- {id: CC3.1, family: Risk Assessment, title: "The entity specifies objectives with sufficient clarity to enable identification of risks."}
	24	- {id: CC3.2, family: Risk Assessment, title: "The entity identifies and analyzes risks to the achievement of its objectives."}
	25	- {id: CC3.3, family: Risk Assessment, title: "The entity considers the potential for fraud in assessing risks."}
	26	- {id: CC3.4, family: Risk Assessment, title: "The entity identifies and assesses changes that could significantly affect internal control."}
	27	# CC4 — Monitoring Activities
	28	- {id: CC4.1, family: Monitoring Activities, title: "The entity selects, develops, and performs ongoing and separate evaluations of internal control."}
	29	- {id: CC4.2, family: Monitoring Activities, title: "The entity evaluates and communicates internal control deficiencies in a timely manner."}
	30	# CC5 — Control Activities
	31	- {id: CC5.1, family: Control Activities, title: "The entity selects and develops control activities that mitigate risks to acceptable levels."}
	32	- {id: CC5.2, family: Control Activities, title: "The entity selects and develops general control activities over technology."}
	33	- {id: CC5.3, family: Control Activities, title: "The entity deploys control activities through policies and procedures."}
	34	# CC6 — Logical and Physical Access Controls
	35	- {id: CC6.1, family: Logical and Physical Access Controls, title: "The entity implements logical access security software, infrastructure, and architectures over protected assets."}
	36	- {id: CC6.2, family: Logical and Physical Access Controls, title: "The entity registers and authorizes new users before granting access, and removes access when appropriate."}
	37	- {id: CC6.3, family: Logical and Physical Access Controls, title: "The entity authorizes, modifies, or removes access based on roles and least privilege."}
	38	- {id: CC6.4, family: Logical and Physical Access Controls, title: "The entity restricts physical access to facilities and protected information assets."}
	39	- {id: CC6.5, family: Logical and Physical Access Controls, title: "The entity discontinues logical and physical protections over assets only after the ability to read data has been removed."}
	40	- {id: CC6.6, family: Logical and Physical Access Controls, title: "The entity implements logical access security measures against threats from outside its system boundaries."}
	41	- {id: CC6.7, family: Logical and Physical Access Controls, title: "The entity restricts the transmission, movement, and removal of information to authorized users and processes."}
	42	- {id: CC6.8, family: Logical and Physical Access Controls, title: "The entity implements controls to prevent or detect and act upon unauthorized or malicious software."}
	43	# CC7 — System Operations
	44	- {id: CC7.1, family: System Operations, title: "The entity uses detection and monitoring procedures to identify configuration changes and new vulnerabilities."}
	45	- {id: CC7.2, family: System Operations, title: "The entity monitors system components for anomalies indicative of malicious acts or errors."}
	46	- {id: CC7.3, family: System Operations, title: "The entity evaluates security events to determine whether they could or did result in a failure to meet objectives."}
	47	- {id: CC7.4, family: System Operations, title: "The entity responds to identified security incidents through a defined program."}
	48	- {id: CC7.5, family: System Operations, title: "The entity identifies, develops, and implements activities to recover from security incidents."}
	49	# CC8 — Change Management
	50	- {id: CC8.1, family: Change Management, title: "The entity authorizes, designs, develops, tests, approves, and implements changes to infrastructure, data, and software."}
	51	# CC9 — Risk Mitigation
	52	- {id: CC9.1, family: Risk Mitigation, title: "The entity identifies, selects, and develops risk mitigation activities for disruptions."}
	53	- {id: CC9.2, family: Risk Mitigation, title: "The entity assesses and manages risks associated with vendors and business partners."}
	54	# Availability category
	55	- {id: A1.1, family: Availability, title: "The entity maintains, monitors, and evaluates current processing capacity to meet demand."}
	56	- {id: A1.2, family: Availability, title: "The entity authorizes, designs, and implements environmental protections, backup, and recovery infrastructure."}
	57	- {id: A1.3, family: Availability, title: "The entity tests recovery plan procedures supporting system recovery."}
	58	# Confidentiality category
	59	- {id: C1.1, family: Confidentiality, title: "The entity identifies and maintains confidential information to meet its objectives related to confidentiality."}
	60	- {id: C1.2, family: Confidentiality, title: "The entity disposes of confidential information to meet its objectives related to confidentiality."}
	61	# Processing Integrity category
	62	- {id: PI1.1, family: Processing Integrity, title: "The entity obtains or generates, uses, and communicates relevant, quality information about processing objectives, including product and service specifications."}
	63	- {id: PI1.2, family: Processing Integrity, title: "The entity implements policies and procedures over system inputs, including controls over completeness and accuracy, to meet its objectives."}
	64	- {id: PI1.3, family: Processing Integrity, title: "The entity implements policies and procedures over system processing to result in products, services, and reporting that meet its objectives."}
	65	- {id: PI1.4, family: Processing Integrity, title: "The entity implements policies and procedures to make available or deliver output completely, accurately, and in a timely manner to meet its objectives."}
	66	- {id: PI1.5, family: Processing Integrity, title: "The entity implements policies and procedures to store inputs, items in processing, and outputs completely, accurately, and in a timely manner to meet its objectives."}
	67	# Privacy category
	68	- {id: P1.1, family: Privacy, title: "The entity provides notice to data subjects about its privacy practices to meet its objectives related to privacy."}
	69	- {id: P2.1, family: Privacy, title: "The entity communicates choices about the collection, use, retention, disclosure, and disposal of personal information, and obtains consent, to meet its privacy objectives."}
	70	- {id: P3.1, family: Privacy, title: "Personal information is collected consistent with the entity's objectives related to privacy."}
	71	- {id: P3.2, family: Privacy, title: "For information requiring explicit consent, the entity communicates the need for and obtains consent prior to collection of personal information."}
	72	- {id: P4.1, family: Privacy, title: "The entity limits the use of personal information to the purposes identified in its objectives related to privacy."}
	73	- {id: P4.2, family: Privacy, title: "The entity retains personal information consistent with its objectives related to privacy."}
	74	- {id: P4.3, family: Privacy, title: "The entity securely disposes of personal information to meet its objectives related to privacy."}
	75	- {id: P5.1, family: Privacy, title: "The entity grants data subjects the ability to access their stored personal information for review and, upon request, provides copies, to meet its privacy objectives."}
	76	- {id: P5.2, family: Privacy, title: "The entity corrects, amends, or appends personal information based on data subject input and communicates it to third parties, to meet its privacy objectives."}
	77	- {id: P6.1, family: Privacy, title: "The entity discloses personal information to third parties only with the explicit consent of data subjects and consistent with its privacy objectives."}
	78	- {id: P6.2, family: Privacy, title: "The entity creates and retains a complete, accurate, and timely record of authorized disclosures of personal information."}
	79	- {id: P6.3, family: Privacy, title: "The entity creates and retains a complete, accurate, and timely record of detected or reported unauthorized disclosures of personal information."}
	80	- {id: P6.4, family: Privacy, title: "The entity obtains privacy commitments from vendors and other third parties who have access to personal information, to meet its privacy objectives."}
	81	- {id: P6.5, family: Privacy, title: "The entity obtains commitments from vendors and third parties to notify it of actual or suspected unauthorized disclosures of personal information."}
	82	- {id: P6.6, family: Privacy, title: "The entity provides notification of breaches and incidents of unauthorized disclosure of personal information to affected data subjects, regulators, and others."}
	83	- {id: P6.7, family: Privacy, title: "The entity provides data subjects with an accounting of the personal information held and disclosures made, upon request."}
	84	- {id: P7.1, family: Privacy, title: "The entity collects and maintains accurate, up-to-date, complete, and relevant personal information to meet its privacy objectives."}
	85	- {id: P8.1, family: Privacy, title: "The entity implements a process for receiving, addressing, resolving, and communicating the resolution of privacy inquiries, complaints, and disputes."}

control_coverage/cli.py added +263

@@ -0,0 +1,263 @@
	1	"""Command-line entry point for control-coverage."""
	2
	3	from __future__ import annotations
	4
	5	import argparse
	6	import sys
	7	from datetime import datetime, timezone
	8	from pathlib import Path
	9
	10	from . import __version__, catalog, corpus, reporters, scope
	11	from .coverage import evaluate
	12
	13
	14	def _parse_args(argv: list[str]) -> argparse.Namespace:
	15	parser = argparse.ArgumentParser(
	16	prog="control-coverage",
	17	description=(
	18	"Score an evidence corpus against complete framework catalogs: what "
	19	"share of each framework does the evidence address, and which controls "
	20	"are blind spots no finding touches?"
	21	),
	22	)
	23	parser.add_argument(
	24	"reports",
	25	nargs="+",
	26	help="audit-report JSON files, and/or directories containing them",
	27	)
	28	parser.add_argument(
	29	"--framework",
	30	help=(
	31	"comma-separated frameworks to evaluate (e.g. SOC2,ISO,NIST). "
	32	"Default: every framework the corpus cites, or the scope file's list."
	33	),
	34	)
	35	parser.add_argument(
	36	"--scope",
	37	help="path to a scope / Statement of Applicability YAML (marks exclusions)",
	38	)
	39	parser.add_argument(
	40	"--subject",
	41	help="name for the subject of this corpus (overrides the scope file)",
	42	)
	43	parser.add_argument(
	44	"--format",
	45	default="md",
	46	help="comma-separated output formats: md, html, json, soa (default: md)",
	47	)
	48	parser.add_argument(
	49	"--out",
	50	help="directory to write reports into (default: print the first format to stdout)",
	51	)
	52	parser.add_argument(
	53	"--blind-spots",
	54	action="store_true",
	55	help="print only the unaddressed in-scope controls, then exit",
	56	)
	57	parser.add_argument(
	58	"--baseline",
	59	metavar="PATH",
	60	help=(
	61	"trend mode: an earlier corpus (file or directory) to compare against. "
	62	"Reports how coverage moved — what improved, regressed, was gained or lost."
	63	),
	64	)
	65	parser.add_argument(
	66	"--crosswalk",
	67	action="store_true",
	68	help=(
	69	"crosswalk mode: show which controls each piece of evidence supports "
	70	"across frameworks, and the minimal evidence set that covers them all"
	71	),
	72	)
	73	parser.add_argument(
	74	"--fail-under",
	75	type=float,
	76	metavar="PCT",
	77	help="exit non-zero if any framework's coverage %% is below PCT (CI gate)",
	78	)
	79	parser.add_argument(
	80	"--fail-on-regression",
	81	action="store_true",
	82	help="in trend mode, exit non-zero if any control regressed or lost coverage",
	83	)
	84	parser.add_argument("--version", action="version", version=f"control-coverage {__version__}")
	85	return parser.parse_args(argv)
	86
	87
	88	def _select_frameworks(args, observations, scp) -> list[str]:
	89	"""Decide which framework catalogs to load, in priority order."""
	90	if args.framework:
	91	return [f.strip() for f in args.framework.split(",") if f.strip()]
	92	if scp.frameworks:
	93	return scp.frameworks
	94	# Infer from the corpus: every framework prefix the observations cite.
	95	cited = sorted({o.control.split(":", 1)[0] for o in observations if ":" in o.control})
	96	if not cited:
	97	raise SystemExit(
	98	"error: could not infer frameworks from the corpus. Pass --framework."
	99	)
	100	return cited
	101
	102
	103	def _print_blind_spots(report) -> None:
	104	total = 0
	105	for fc in report.frameworks:
	106	spots = fc.blind_spots
	107	if not spots:
	108	continue
	109	print(f"{fc.catalog.name} — {len(spots)} unaddressed:")
	110	for r in spots:
	111	print(f" {r.control.code} {r.control.title}")
	112	total += len(spots)
	113	print(f"\n{total} in-scope control(s) unaddressed across {len(report.frameworks)} framework(s).")
	114
	115
	116	def _now() -> str:
	117	return datetime.now(timezone.utc).strftime("%Y-%m-%d %H:%M:%S UTC")
	118
	119
	120	def _build_report(paths, args, scp, names, subject):
	121	"""Load a corpus from paths and evaluate it into a CoverageReport."""
	122	try:
	123	observations = corpus.load_corpus(paths)
	124	except (ValueError, FileNotFoundError, OSError) as exc:
	125	raise SystemExit(f"error: {exc}") from None
	126	catalogs = catalog.load_frameworks(names)
	127	return evaluate(catalogs, observations, scope=scp, subject=subject, generated_at=_now())
	128
	129
	130	def main(argv: list[str] \| None = None) -> int:
	131	args = _parse_args(sys.argv[1:] if argv is None else argv)
	132
	133	if args.crosswalk and args.baseline:
	134	raise SystemExit("error: --crosswalk and --baseline cannot be combined")
	135
	136	try:
	137	observations = corpus.load_corpus(args.reports)
	138	except (ValueError, FileNotFoundError, OSError) as exc:
	139	raise SystemExit(f"error: {exc}") from None
	140
	141	scp = scope.load(args.scope) if args.scope else scope.empty()
	142
	143	try:
	144	names = _select_frameworks(args, observations, scp)
	145	catalogs = catalog.load_frameworks(names)
	146	except ValueError as exc:
	147	raise SystemExit(f"error: {exc}") from None
	148
	149	subject = args.subject or scp.subject
	150	report = evaluate(catalogs, observations, scope=scp, subject=subject, generated_at=_now())
	151
	152	if args.baseline:
	153	return _trend_mode(args, scp, names, subject, report)
	154
	155	if args.crosswalk:
	156	return _crosswalk_mode(args, report)
	157
	158	if args.blind_spots:
	159	_print_blind_spots(report)
	160	return _exit_code(report, args.fail_under)
	161
	162	formats = [f.strip() for f in args.format.split(",") if f.strip()]
	163	if args.out:
	164	out_dir = Path(args.out)
	165	out_dir.mkdir(parents=True, exist_ok=True)
	166	stem = _slug(subject) or "coverage"
	167	for fmt in formats:
	168	ext = reporters.EXTENSIONS.get(fmt, fmt)
	169	name = "soa" if fmt == "soa" else "coverage"
	170	path = out_dir / f"{name}.{ext}" if fmt == "soa" else out_dir / f"{stem}.{ext}"
	171	path.write_text(reporters.render(report, fmt), encoding="utf-8")
	172	print(f"wrote {path}")
	173	else:
	174	# Print the first requested format to stdout.
	175	print(reporters.render(report, formats[0]), end="")
	176
	177	return _exit_code(report, args.fail_under)
	178
	179
	180	def _trend_mode(args, scp, names, subject, current) -> int:
	181	from . import trend
	182
	183	baseline = _build_report([args.baseline], args, scp, names, subject)
	184	tr = trend.compare(baseline, current)
	185
	186	formats = [f.strip() for f in args.format.split(",") if f.strip()]
	187	renderers = {
	188	"md": trend.render_markdown,
	189	"html": trend.render_html,
	190	"json": trend.render_json,
	191	}
	192	unknown = [f for f in formats if f not in renderers]
	193	if unknown:
	194	raise SystemExit(f"error: trend mode supports md, html, and json, not: {', '.join(unknown)}")
	195
	196	if args.out:
	197	out_dir = Path(args.out)
	198	out_dir.mkdir(parents=True, exist_ok=True)
	199	for fmt in formats:
	200	path = out_dir / f"trend.{fmt}"
	201	path.write_text(renderers[fmt](tr), encoding="utf-8")
	202	print(f"wrote {path}")
	203	else:
	204	print(renderers[formats[0]](tr), end="")
	205
	206	if args.fail_on_regression and tr.total_regressions:
	207	print(
	208	f"trend gate: {tr.total_regressions} control(s) regressed or lost coverage",
	209	file=sys.stderr,
	210	)
	211	return 1
	212	return 0
	213
	214
	215	def _crosswalk_mode(args, report) -> int:
	216	from . import crosswalk
	217
	218	xw = crosswalk.build(report)
	219	formats = [f.strip() for f in args.format.split(",") if f.strip()]
	220	renderers = {
	221	"md": crosswalk.render_markdown,
	222	"html": crosswalk.render_html,
	223	"json": crosswalk.render_json,
	224	}
	225	unknown = [f for f in formats if f not in renderers]
	226	if unknown:
	227	raise SystemExit(
	228	f"error: crosswalk mode supports md, html, and json, not: {', '.join(unknown)}"
	229	)
	230
	231	if args.out:
	232	out_dir = Path(args.out)
	233	out_dir.mkdir(parents=True, exist_ok=True)
	234	for fmt in formats:
	235	path = out_dir / f"crosswalk.{fmt}"
	236	path.write_text(renderers[fmt](xw), encoding="utf-8")
	237	print(f"wrote {path}")
	238	else:
	239	print(renderers[formats[0]](xw), end="")
	240	return 0
	241
	242
	243	def _exit_code(report, fail_under: float \| None) -> int:
	244	if fail_under is None:
	245	return 0
	246	below = [fc for fc in report.frameworks if fc.coverage_pct < fail_under]
	247	if below:
	248	for fc in below:
	249	print(
	250	f"coverage gate: {fc.catalog.framework} at {fc.coverage_pct}% "
	251	f"is below {fail_under}%",
	252	file=sys.stderr,
	253	)
	254	return 1
	255	return 0
	256
	257
	258	def _slug(text: str) -> str:
	259	return "".join(c if c.isalnum() else "-" for c in text.lower()).strip("-")
	260
	261
	262	if __name__ == "__main__":
	263	raise SystemExit(main())

control_coverage/corpus.py added +88

@@ -0,0 +1,88 @@
	1	"""Load an evidence corpus from audit-report JSON reports.
	2
	3	audit-report emits one JSON document per evidence package. Each finding carries
	4	the controls it maps to and a status::
	5
	6	{"findings": [
	7	{"id": "github.org.require-2fa", "status": "pass", "severity": "high",
	8	"controls": ["SOC2:CC6.1", "ISO:A.5.17", "NIST:IA-2"], ...},
	9	...
	10	]}
	11
	12	A corpus is any number of these reports — typically one per platform and date
	13	(AWS, GitHub, GitLab…) — flattened into a list of :class:`Observation`, one per
	14	(finding, control) pair. Coverage is then computed by joining observations onto a
	15	framework catalog.
	16	"""
	17
	18	from __future__ import annotations
	19
	20	import json
	21	from dataclasses import dataclass
	22	from pathlib import Path
	23
	24	# Statuses as emitted by audit-report's engine.
	25	PASS = "pass"
	26	FAIL = "fail"
	27	NOT_APPLICABLE = "not_applicable"
	28
	29
	30	@dataclass(frozen=True)
	31	class Observation:
	32	"""One finding's bearing on one control, with provenance."""
	33
	34	control: str # "FRAMEWORK:ID"
	35	status: str # pass \| fail \| not_applicable
	36	rule_id: str
	37	title: str
	38	severity: str
	39	source: str # report file / package name the finding came from
	40
	41
	42	def _iter_report(doc: dict, source: str):
	43	for f in doc.get("findings", []):
	44	status = f.get("status", NOT_APPLICABLE)
	45	rule_id = f.get("id", "")
	46	title = f.get("title", "")
	47	severity = f.get("severity", "medium")
	48	for control in f.get("controls", []):
	49	yield Observation(
	50	control=control,
	51	status=status,
	52	rule_id=rule_id,
	53	title=title,
	54	severity=severity,
	55	source=source,
	56	)
	57
	58
	59	def load_report(path: str \| Path) -> list[Observation]:
	60	"""Load observations from a single audit-report JSON file."""
	61	p = Path(path)
	62	doc = json.loads(p.read_text(encoding="utf-8"))
	63	# Prefer the package name audit-report records; fall back to the file name.
	64	source = doc.get("source_package") or p.name
	65	return list(_iter_report(doc, source))
	66
	67
	68	def _expand(paths: list[str \| Path]) -> list[Path]:
	69	"""Resolve inputs: JSON files pass through; directories contribute their *.json."""
	70	resolved: list[Path] = []
	71	for raw in paths:
	72	p = Path(raw)
	73	if p.is_dir():
	74	resolved.extend(sorted(p.glob("*.json")))
	75	else:
	76	resolved.append(p)
	77	return resolved
	78
	79
	80	def load_corpus(paths: list[str \| Path]) -> list[Observation]:
	81	"""Load and flatten observations from files and/or directories of reports."""
	82	files = _expand(paths)
	83	if not files:
	84	raise ValueError("no audit-report JSON files found in the given paths")
	85	observations: list[Observation] = []
	86	for f in files:
	87	observations.extend(load_report(f))
	88	return observations

control_coverage/coverage.py added +179

@@ -0,0 +1,179 @@
	1	"""Compute control coverage: join an evidence corpus onto framework catalogs.
	2
	3	For every control in a catalog we assign one assurance state:
	4
	5	* ``supported`` — in scope, at least one mapped finding passes and none fail.
	6	* ``failing`` — in scope, at least one mapped finding fails.
	7	* ``asserted`` — in scope, findings map here but their data was absent
	8	(``not_applicable``): evidence was attempted, not obtained.
	9	* ``unaddressed`` — in scope, no finding maps here at all. The blind spot.
	10	* ``out_of_scope``— excluded by the scope file, with a recorded justification.
	11
	12	When several findings touch one control the worst wins: a single failure makes the
	13	control ``failing`` regardless of how many others pass. Coverage is then a headline
	14	number the evidence-first tools cannot produce — of everything a framework
	15	requires, how much the corpus even looks at, and how much it supports.
	16	"""
	17
	18	from __future__ import annotations
	19
	20	from dataclasses import dataclass, field
	21
	22	from .catalog import Catalog, Control
	23	from .corpus import FAIL, PASS, Observation
	24
	25	SUPPORTED = "supported"
	26	FAILING = "failing"
	27	ASSERTED = "asserted"
	28	UNADDRESSED = "unaddressed"
	29	OUT_OF_SCOPE = "out_of_scope"
	30
	31	# Order states appear in reports and roll up in summaries (most urgent first).
	32	STATE_ORDER = [FAILING, UNADDRESSED, ASSERTED, SUPPORTED, OUT_OF_SCOPE]
	33
	34	# States that count as the control being "addressed" by the corpus at all.
	35	_ADDRESSED = {SUPPORTED, FAILING, ASSERTED}
	36
	37
	38	@dataclass
	39	class ControlResult:
	40	"""One control's assurance state and the evidence behind it."""
	41
	42	control: Control
	43	state: str
	44	observations: list[Observation] = field(default_factory=list)
	45	owner: str = ""
	46	exclusion_reason: str = ""
	47
	48	@property
	49	def addressed(self) -> bool:
	50	return self.state in _ADDRESSED
	51
	52
	53	def _state_for(observations: list[Observation]) -> str:
	54	"""Worst-wins resolution of a control's state from its observations."""
	55	if not observations:
	56	return UNADDRESSED
	57	statuses = {o.status for o in observations}
	58	if FAIL in statuses:
	59	return FAILING
	60	if PASS in statuses:
	61	return SUPPORTED
	62	return ASSERTED # only not_applicable observations remain
	63
	64
	65	@dataclass
	66	class FrameworkCoverage:
	67	"""Coverage of one framework catalog by the corpus."""
	68
	69	catalog: Catalog
	70	results: list[ControlResult]
	71
	72	def by_state(self, state: str) -> list[ControlResult]:
	73	return [r for r in self.results if r.state == state]
	74
	75	@property
	76	def counts(self) -> dict[str, int]:
	77	counts = {s: 0 for s in STATE_ORDER}
	78	for r in self.results:
	79	counts[r.state] += 1
	80	return counts
	81
	82	@property
	83	def in_scope(self) -> int:
	84	return sum(1 for r in self.results if r.state != OUT_OF_SCOPE)
	85
	86	@property
	87	def addressed(self) -> int:
	88	return sum(1 for r in self.results if r.addressed)
	89
	90	@property
	91	def supported(self) -> int:
	92	return sum(1 for r in self.results if r.state == SUPPORTED)
	93
	94	@property
	95	def coverage_pct(self) -> float:
	96	"""Share of in-scope controls the corpus touches at all (0–100)."""
	97	return round(100 * self.addressed / self.in_scope, 1) if self.in_scope else 0.0
	98
	99	@property
	100	def assured_pct(self) -> float:
	101	"""Share of in-scope controls that are supported and not failing (0–100)."""
	102	return round(100 * self.supported / self.in_scope, 1) if self.in_scope else 0.0
	103
	104	@property
	105	def blind_spots(self) -> list[ControlResult]:
	106	"""In-scope controls no finding touches — the headline gap list."""
	107	return self.by_state(UNADDRESSED)
	108
	109
	110	@dataclass
	111	class CoverageReport:
	112	"""Coverage across every requested framework, plus corpus-wide diagnostics."""
	113
	114	subject: str
	115	generated_at: str
	116	frameworks: list[FrameworkCoverage]
	117	# Control codes cited by the corpus that no loaded catalog defines. These are
	118	# typos, renamed controls, or controls outside the bundled catalogs — either
	119	# way, evidence pointing at nothing is worth surfacing.
	120	orphan_codes: list[str] = field(default_factory=list)
	121	source_count: int = 0
	122
	123
	124	def _observations_by_control(observations: list[Observation]) -> dict[str, list[Observation]]:
	125	grouped: dict[str, list[Observation]] = {}
	126	for obs in observations:
	127	grouped.setdefault(obs.control, []).append(obs)
	128	return grouped
	129
	130
	131	def evaluate(
	132	catalogs: list[Catalog],
	133	observations: list[Observation],
	134	scope=None,
	135	subject: str = "",
	136	generated_at: str = "",
	137	) -> CoverageReport:
	138	"""Produce a :class:`CoverageReport` from catalogs, a corpus, and a scope."""
	139	grouped = _observations_by_control(observations)
	140	catalog_codes: set[str] = set()
	141	loaded_frameworks = {cat.framework for cat in catalogs}
	142
	143	frameworks: list[FrameworkCoverage] = []
	144	for cat in catalogs:
	145	catalog_codes \|= cat.codes()
	146	results: list[ControlResult] = []
	147	for control in cat.controls:
	148	code = control.code
	149	obs = grouped.get(code, [])
	150	owner = scope.owner(code) if scope else ""
	151	if scope and scope.excluded(code):
	152	results.append(
	153	ControlResult(control, OUT_OF_SCOPE, obs, owner, scope.reason(code))
	154	)
	155	elif scope and scope.family_excluded(cat.framework, control.family):
	156	reason = scope.family_reason(cat.framework, control.family)
	157	results.append(ControlResult(control, OUT_OF_SCOPE, obs, owner, reason))
	158	else:
	159	results.append(ControlResult(control, _state_for(obs), obs, owner))
	160	frameworks.append(FrameworkCoverage(cat, results))
	161
	162	# A code is an orphan only when its framework is loaded but the catalog
	163	# does not define it — a typo or a renamed control. Codes for frameworks we
	164	# did not load this run are simply out of scope, not orphans.
	165	cited = {o.control for o in observations}
	166	orphans = sorted(
	167	code
	168	for code in cited
	169	if code.split(":", 1)[0] in loaded_frameworks and code not in catalog_codes
	170	)
	171	sources = {o.source for o in observations}
	172
	173	return CoverageReport(
	174	subject=subject,
	175	generated_at=generated_at,
	176	frameworks=frameworks,
	177	orphan_codes=orphans,
	178	source_count=len(sources),
	179	)

control_coverage/crosswalk.py added +291

@@ -0,0 +1,291 @@
	1	"""Crosswalk — which controls each piece of evidence supports, across frameworks.
	2
	3	One check is rarely worth one control. Enforced 2FA is evidence for SOC 2 CC6.1,
	4	ISO A.5.17, and NIST IA-2 at once. This module inverts the coverage result to show
	5	that leverage: for every check (rule) in the corpus, the set of controls it
	6	addresses and the frameworks it spans.
	7
	8	It then answers a practical question auditors and evidence-owners both ask — *what
	9	is the smallest set of checks that still covers everything?* — with a greedy
	10	set-cover over the addressed controls. The result is an ordered "minimal evidence
	11	set": collect these few checks and you have touched every control the full corpus
	12	touches, which is what you want when scoping a walkthrough or a sample.
	13
	14	"Addressed" here matches the coverage engine: a control any finding maps to,
	15	whatever the finding's outcome.
	16	"""
	17
	18	from __future__ import annotations
	19
	20	from dataclasses import dataclass, field
	21
	22	from .coverage import CoverageReport
	23
	24
	25	@dataclass
	26	class EvidenceItem:
	27	"""One check and the controls it supports across frameworks."""
	28
	29	rule_id: str
	30	title: str
	31	controls: list[str] # full FRAMEWORK:ID codes, sorted
	32	frameworks: list[str] # framework short codes it spans, sorted
	33
	34	@property
	35	def count(self) -> int:
	36	return len(self.controls)
	37
	38
	39	@dataclass
	40	class CoverStep:
	41	"""One pick in the greedy minimal-evidence set."""
	42
	43	rule_id: str
	44	new_controls: int # controls this pick added that were not yet covered
	45	cumulative: int
	46	cumulative_pct: float
	47
	48
	49	@dataclass
	50	class Crosswalk:
	51	subject: str
	52	generated_at: str
	53	items: list[EvidenceItem] = field(default_factory=list)
	54	cover: list[CoverStep] = field(default_factory=list)
	55	universe_size: int = 0
	56
	57	@property
	58	def multi_framework(self) -> list[EvidenceItem]:
	59	"""Checks that earn coverage in more than one framework at once."""
	60	return [i for i in self.items if len(i.frameworks) > 1]
	61
	62
	63	def build(report: CoverageReport) -> Crosswalk:
	64	"""Invert a coverage report into a crosswalk and a minimal evidence set."""
	65	rule_controls: dict[str, set[str]] = {}
	66	rule_title: dict[str, str] = {}
	67	rule_frameworks: dict[str, set[str]] = {}
	68	universe: set[str] = set()
	69
	70	for fc in report.frameworks:
	71	for r in fc.results:
	72	if not r.addressed:
	73	continue
	74	code = r.control.code
	75	universe.add(code)
	76	for obs in r.observations:
	77	if not obs.rule_id:
	78	continue
	79	rule_controls.setdefault(obs.rule_id, set()).add(code)
	80	rule_title.setdefault(obs.rule_id, obs.title)
	81	rule_frameworks.setdefault(obs.rule_id, set()).add(fc.catalog.framework)
	82
	83	items = [
	84	EvidenceItem(
	85	rule_id=rid,
	86	title=rule_title.get(rid, ""),
	87	controls=sorted(codes),
	88	frameworks=sorted(rule_frameworks.get(rid, set())),
	89	)
	90	for rid, codes in rule_controls.items()
	91	]
	92	# Most leverage first; rule id breaks ties for stable output.
	93	items.sort(key=lambda i: (-i.count, i.rule_id))
	94
	95	cover = _greedy_cover(rule_controls, universe)
	96	return Crosswalk(
	97	subject=report.subject,
	98	generated_at=report.generated_at,
	99	items=items,
	100	cover=cover,
	101	universe_size=len(universe),
	102	)
	103
	104
	105	def _greedy_cover(rule_controls: dict[str, set[str]], universe: set[str]) -> list[CoverStep]:
	106	remaining = set(universe)
	107	pool = {rid: set(codes) for rid, codes in rule_controls.items()}
	108	total = len(universe) or 1
	109	steps: list[CoverStep] = []
	110
	111	while remaining:
	112	best_rule, best_gain = None, 0
	113	for rid in sorted(pool):
	114	gain = len(pool[rid] & remaining)
	115	if gain > best_gain:
	116	best_rule, best_gain = rid, gain
	117	if not best_rule: # nothing left can cover the remainder
	118	break
	119	remaining -= pool[best_rule]
	120	del pool[best_rule]
	121	cumulative = len(universe) - len(remaining)
	122	steps.append(
	123	CoverStep(
	124	rule_id=best_rule,
	125	new_controls=best_gain,
	126	cumulative=cumulative,
	127	cumulative_pct=round(100 * cumulative / total, 1),
	128	)
	129	)
	130	return steps
	131
	132
	133	# --- renderers -------------------------------------------------------------
	134
	135
	136	def to_dict(xw: Crosswalk) -> dict:
	137	return {
	138	"subject": xw.subject,
	139	"generated_at": xw.generated_at,
	140	"universe_size": xw.universe_size,
	141	"minimal_evidence_set": [
	142	{
	143	"rule_id": s.rule_id,
	144	"new_controls": s.new_controls,
	145	"cumulative": s.cumulative,
	146	"cumulative_pct": s.cumulative_pct,
	147	}
	148	for s in xw.cover
	149	],
	150	"evidence": [
	151	{
	152	"rule_id": i.rule_id,
	153	"title": i.title,
	154	"frameworks": i.frameworks,
	155	"controls": i.controls,
	156	"count": i.count,
	157	}
	158	for i in xw.items
	159	],
	160	}
	161
	162
	163	def render_json(xw: Crosswalk) -> str:
	164	import json
	165
	166	return json.dumps(to_dict(xw), indent=2, sort_keys=False) + "\n"
	167
	168
	169	def render_markdown(xw: Crosswalk) -> str:
	170	out: list[str] = []
	171	out.append(f"# Evidence Crosswalk — {xw.subject or 'Evidence corpus'}")
	172	out.append("")
	173	out.append(f"- Generated: {xw.generated_at}")
	174	out.append(f"- Addressed controls: {xw.universe_size}")
	175	out.append(f"- Checks in corpus: {len(xw.items)}")
	176	out.append(f"- Checks spanning multiple frameworks: {len(xw.multi_framework)}")
	177	out.append("")
	178
	179	out.append("## Minimal evidence set")
	180	out.append("")
	181	if xw.cover:
	182	out.append(
	183	f"The {len(xw.cover)} check(s) below cover all {xw.universe_size} addressed "
	184	"controls — the smallest set that touches everything the full corpus does."
	185	)
	186	out.append("")
	187	out.append("\| # \| Check \| New controls \| Cumulative \| % of addressed \|")
	188	out.append("\| ---: \| --- \| ---: \| ---: \| ---: \|")
	189	for n, s in enumerate(xw.cover, 1):
	190	out.append(
	191	f"\| {n} \| `{s.rule_id}` \| +{s.new_controls} \| {s.cumulative} \| {s.cumulative_pct}% \|"
	192	)
	193	else:
	194	out.append("_No addressed controls to cover._")
	195	out.append("")
	196
	197	out.append("## Evidence leverage")
	198	out.append("")
	199	out.append("Each check and the controls it supports, most leverage first.")
	200	out.append("")
	201	out.append("\| Check \| Frameworks \| # \| Controls \|")
	202	out.append("\| --- \| --- \| ---: \| --- \|")
	203	for i in xw.items:
	204	codes = ", ".join(f"`{c}`" for c in i.controls)
	205	fws = ", ".join(i.frameworks)
	206	out.append(f"\| `{i.rule_id}` \| {fws} \| {i.count} \| {codes} \|")
	207	out.append("")
	208
	209	return "\n".join(out).rstrip() + "\n"
	210
	211
	212	_XW_CSS = """
	213	.fw { display: inline-block; font-size: .7rem; font-weight: 700; letter-spacing: .02em;
	214	padding: .08rem .4rem; border-radius: 4px; margin-right: .25rem; background: #eef; color: #33488c; }
	215	.track { position: relative; background: #eee; border-radius: 4px; height: 1rem; min-width: 5rem; }
	216	.track > span { position: absolute; left: 0; top: 0; bottom: 0; background: #35b866; border-radius: 4px; }
	217	.codes code { font-size: .78rem; }
	218	@media (prefers-color-scheme: dark) {
	219	.fw { background: #22243a; color: #9fb0f0; }
	220	.track { background: #26272b; }
	221	}
	222	"""
	223
	224
	225	def render_html(xw: Crosswalk) -> str:
	226	from html import escape
	227
	228	from .reporters.html import CSS
	229
	230	title = xw.subject or "Evidence corpus"
	231	body = [
	232	"<!doctype html><html lang='en'><head><meta charset='utf-8'>",
	233	"<meta name='viewport' content='width=device-width, initial-scale=1'>",
	234	f"<title>Evidence Crosswalk — {escape(title)}</title>",
	235	f"<style>{CSS}{_XW_CSS}</style></head><body><main>",
	236	f"<h1>Evidence Crosswalk — {escape(title)}</h1>",
	237	(
	238	f'<p class="meta">Generated {escape(xw.generated_at)} · '
	239	f"{xw.universe_size} addressed control(s) · {len(xw.items)} check(s) · "
	240	f"{len(xw.multi_framework)} spanning multiple frameworks</p>"
	241	),
	242	"<h2>Minimal evidence set</h2>",
	243	]
	244	if xw.cover:
	245	body.append(
	246	f"<p>The {len(xw.cover)} check(s) below cover all {xw.universe_size} addressed "
	247	"controls — the smallest set that touches everything the full corpus does.</p>"
	248	)
	249	body.append(
	250	"<table><thead><tr><th class='num'>#</th><th>Check</th>"
	251	"<th class='num'>New</th><th class='num'>Cumulative</th>"
	252	"<th>% of addressed</th></tr></thead><tbody>"
	253	)
	254	for n, s in enumerate(xw.cover, 1):
	255	body.append(
	256	"<tr>"
	257	f'<td class="num">{n}</td>'
	258	f"<td><code>{escape(s.rule_id)}</code></td>"
	259	f'<td class="num">+{s.new_controls}</td>'
	260	f'<td class="num">{s.cumulative}</td>'
	261	f'<td><div class="track" title="{s.cumulative_pct}%">'
	262	f'<span style="width:{s.cumulative_pct:.1f}%"></span></div> {s.cumulative_pct}%</td>'
	263	"</tr>"
	264	)
	265	body.append("</tbody></table>")
	266	else:
	267	body.append("<p>No addressed controls to cover.</p>")
	268
	269	body.append("<h2>Evidence leverage</h2>")
	270	body.append("<p>Each check and the controls it supports, most leverage first.</p>")
	271	body.append(
	272	"<table><thead><tr><th>Check</th><th>Frameworks</th><th class='num'>#</th>"
	273	"<th>Controls</th></tr></thead><tbody>"
	274	)
	275	for i in xw.items:
	276	fws = "".join(f'<span class="fw">{escape(f)}</span>' for f in i.frameworks)
	277	codes = ", ".join(f"<code>{escape(c)}</code>" for c in i.controls)
	278	body.append(
	279	"<tr>"
	280	f"<td><code>{escape(i.rule_id)}</code></td>"
	281	f"<td>{fws}</td>"
	282	f'<td class="num">{i.count}</td>'
	283	f'<td class="codes">{codes}</td>'
	284	"</tr>"
	285	)
	286	body.append("</tbody></table>")
	287	body.append(
	288	"<footer>Generated by control-coverage · Audit Labs. Evidence, not a verdict.</footer>"
	289	)
	290	body.append("</main></body></html>")
	291	return "".join(body)

control_coverage/reporters/__init__.py added +26

@@ -0,0 +1,26 @@
	1	"""Renderers for a coverage report: Markdown, HTML, JSON, and a Statement of Applicability."""
	2
	3	from __future__ import annotations
	4
	5	from ..coverage import CoverageReport
	6	from . import html, json, markdown, soa
	7
	8	_RENDERERS = {
	9	"md": markdown.render,
	10	"markdown": markdown.render,
	11	"html": html.render,
	12	"json": json.render,
	13	"soa": soa.render,
	14	}
	15
	16	# File extension per format (soa is Markdown by default).
	17	EXTENSIONS = {"md": "md", "markdown": "md", "html": "html", "json": "json", "soa": "md"}
	18
	19
	20	def render(report: CoverageReport, fmt: str) -> str:
	21	try:
	22	return _RENDERERS[fmt](report)
	23	except KeyError:
	24	raise ValueError(
	25	f"unknown format '{fmt}'. Choose from: {', '.join(sorted(_RENDERERS))}"
	26	) from None

control_coverage/reporters/html.py added +252

@@ -0,0 +1,252 @@
	1	"""HTML renderer — a self-contained, printable coverage report.
	2
	3	No external assets: all CSS is inlined so the file can be attached to an audit
	4	workpaper and opened anywhere, including offline.
	5	"""
	6
	7	from __future__ import annotations
	8
	9	from html import escape
	10	from typing import TYPE_CHECKING
	11
	12	from ..coverage import (
	13	ASSERTED,
	14	FAILING,
	15	OUT_OF_SCOPE,
	16	SUPPORTED,
	17	UNADDRESSED,
	18	)
	19
	20	if TYPE_CHECKING:
	21	from ..coverage import CoverageReport, FrameworkCoverage
	22
	23	_STATE_LABEL = {
	24	SUPPORTED: "supported",
	25	FAILING: "failing",
	26	ASSERTED: "asserted",
	27	UNADDRESSED: "unaddressed",
	28	OUT_OF_SCOPE: "out of scope",
	29	}
	30	_STATE_CLASS = {
	31	SUPPORTED: "supported",
	32	FAILING: "failing",
	33	ASSERTED: "asserted",
	34	UNADDRESSED: "unaddressed",
	35	OUT_OF_SCOPE: "oos",
	36	}
	37
	38	CSS = """
	39	:root { color-scheme: light dark; }
	40	* { box-sizing: border-box; }
	41	body { font-family: -apple-system, Segoe UI, Roboto, Helvetica, Arial, sans-serif;
	42	margin: 0; padding: 2rem; line-height: 1.5; color: #1a1a1a; background: #fff; }
	43	main { max-width: 64rem; margin: 0 auto; }
	44	h1 { margin: 0 0 .25rem; font-size: 1.6rem; }
	45	h2 { margin: 2rem 0 .75rem; font-size: 1.25rem; border-bottom: 2px solid #e5e5e5; padding-bottom: .25rem; }
	46	h3 { margin: 1.4rem 0 .5rem; font-size: 1.02rem; }
	47	.meta { color: #555; font-size: .9rem; margin: 0 0 1rem; }
	48	.meta code { background: #f2f2f2; padding: .05rem .3rem; border-radius: 3px; }
	49	.note { background: #f7f7f9; border-left: 3px solid #b9b9c6; padding: .6rem .9rem;
	50	font-size: .9rem; color: #444; border-radius: 0 4px 4px 0; }
	51	table { border-collapse: collapse; width: 100%; font-size: .85rem; margin: .5rem 0; }
	52	th, td { border: 1px solid #e0e0e0; padding: .35rem .5rem; text-align: left; vertical-align: top; }
	53	th { background: #f5f5f7; }
	54	td.num, th.num { text-align: right; }
	55	.badge { display: inline-block; font-weight: 700; font-size: .72rem; letter-spacing: .02em;
	56	padding: .12rem .5rem; border-radius: 999px; white-space: nowrap; }
	57	.badge.supported { background: #e5f6ea; color: #1a7f37; }
	58	.badge.failing { background: #fdeaea; color: #c1272d; }
	59	.badge.asserted { background: #fff4e0; color: #a8620a; }
	60	.badge.unaddressed { background: #eceaf6; color: #5b4bb0; }
	61	.badge.oos { background: #eee; color: #666; }
	62	.bar { display: flex; height: 1.1rem; border-radius: 4px; overflow: hidden; margin: .4rem 0 .2rem;
	63	border: 1px solid #ddd; }
	64	.bar > span { display: block; }
	65	.bar .supported { background: #35b866; }
	66	.bar .failing { background: #e2565b; }
	67	.bar .asserted { background: #eaa53c; }
	68	.bar .unaddressed { background: #8877d8; }
	69	.bar .oos { background: #cfcfcf; }
	70	.headline { font-size: 1.5rem; font-weight: 700; }
	71	.headline small { font-size: .85rem; font-weight: 500; color: #666; }
	72	.legend { font-size: .78rem; color: #666; display: flex; flex-wrap: wrap; gap: .8rem; margin: .2rem 0 1rem; }
	73	.legend i { display: inline-block; width: .8rem; height: .8rem; border-radius: 2px; vertical-align: -1px; margin-right: .25rem; }
	74	footer { margin-top: 3rem; font-size: .8rem; color: #888; border-top: 1px solid #eee; padding-top: .75rem; }
	75	@media (prefers-color-scheme: dark) {
	76	body { color: #e6e6e6; background: #16171a; }
	77	h2 { border-color: #333; }
	78	.meta { color: #aaa; } .meta code { background: #26272b; }
	79	.note { background: #1e1f24; border-color: #444; color: #bbb; }
	80	th, td { border-color: #333; } th { background: #202126; }
	81	.headline small, .legend { color: #999; }
	82	.badge.supported { background: #12321d; color: #4ac36a; }
	83	.badge.failing { background: #3a1416; color: #ff6b70; }
	84	.badge.asserted { background: #33260f; color: #e6a94e; }
	85	.badge.unaddressed { background: #211d3a; color: #9d8ef0; }
	86	.badge.oos { background: #26272b; color: #999; }
	87	.bar { border-color: #333; }
	88	footer { border-color: #2a2b30; }
	89	}
	90	"""
	91
	92	_LEGEND_COLORS = {
	93	SUPPORTED: "#35b866",
	94	FAILING: "#e2565b",
	95	ASSERTED: "#eaa53c",
	96	UNADDRESSED: "#8877d8",
	97	OUT_OF_SCOPE: "#cfcfcf",
	98	}
	99
	100
	101	def _badge(state: str) -> str:
	102	return f'<span class="badge {_STATE_CLASS[state]}">{_STATE_LABEL[state]}</span>'
	103
	104
	105	def _bar(fc: FrameworkCoverage) -> str:
	106	counts = fc.counts
	107	total = sum(counts.values()) or 1
	108	segments = []
	109	for state in [SUPPORTED, FAILING, ASSERTED, UNADDRESSED, OUT_OF_SCOPE]:
	110	n = counts[state]
	111	if not n:
	112	continue
	113	pct = 100 * n / total
	114	segments.append(
	115	f'<span class="{_STATE_CLASS[state]}" style="width:{pct:.2f}%" '
	116	f'title="{n} {_STATE_LABEL[state]}"></span>'
	117	)
	118	return '<div class="bar">' + "".join(segments) + "</div>"
	119
	120
	121	def _legend() -> str:
	122	items = []
	123	for state in [SUPPORTED, FAILING, ASSERTED, UNADDRESSED, OUT_OF_SCOPE]:
	124	items.append(
	125	f'<span><i style="background:{_LEGEND_COLORS[state]}"></i>{_STATE_LABEL[state]}</span>'
	126	)
	127	return '<div class="legend">' + "".join(items) + "</div>"
	128
	129
	130	def _checked_by(result) -> str:
	131	if result.state == OUT_OF_SCOPE:
	132	return f"<em>excluded: {escape(result.exclusion_reason)}</em>"
	133	rules = sorted({o.rule_id for o in result.observations if o.rule_id})
	134	return ", ".join(f"<code>{escape(r)}</code>" for r in rules)
	135
	136
	137	def _framework_section(fc: FrameworkCoverage) -> str:
	138	cat = fc.catalog
	139	partial = "" if cat.complete else (
	140	' <small>(partial catalog — coverage is of the shipped subset)</small>'
	141	)
	142	rows = []
	143	for r in fc.results:
	144	rows.append(
	145	"<tr>"
	146	f"<td><strong>{escape(r.control.id)}</strong></td>"
	147	f"<td>{_badge(r.state)}</td>"
	148	f"<td>{escape(r.control.title)}</td>"
	149	f"<td>{_checked_by(r)}</td>"
	150	"</tr>"
	151	)
	152	return (
	153	f"<h2>{escape(cat.name)}{partial}</h2>"
	154	f'<p class="headline">{fc.coverage_pct}% <small>coverage · {fc.addressed}/{fc.in_scope} '
	155	f"in-scope controls addressed · {fc.assured_pct}% assured</small></p>"
	156	f"{_bar(fc)}{_legend()}"
	157	"<table><thead><tr><th>Control</th><th>Status</th><th>Description</th>"
	158	"<th>Checked by</th></tr></thead><tbody>"
	159	+ "".join(rows)
	160	+ "</tbody></table>"
	161	)
	162
	163
	164	def _summary_table(report: CoverageReport) -> str:
	165	rows = []
	166	for fc in report.frameworks:
	167	c = fc.counts
	168	rows.append(
	169	"<tr>"
	170	f"<td>{escape(fc.catalog.name)}</td>"
	171	f'<td class="num">{fc.in_scope}</td>'
	172	f'<td class="num">{fc.addressed}</td>'
	173	f'<td class="num">{fc.supported}</td>'
	174	f'<td class="num">{c[FAILING]}</td>'
	175	f'<td class="num">{len(fc.blind_spots)}</td>'
	176	f'<td class="num">{fc.coverage_pct}%</td>'
	177	f'<td class="num">{fc.assured_pct}%</td>'
	178	"</tr>"
	179	)
	180	return (
	181	"<table><thead><tr><th>Framework</th><th class='num'>In scope</th>"
	182	"<th class='num'>Addressed</th><th class='num'>Supported</th>"
	183	"<th class='num'>Failing</th><th class='num'>Blind spots</th>"
	184	"<th class='num'>Coverage</th><th class='num'>Assured</th></tr></thead><tbody>"
	185	+ "".join(rows)
	186	+ "</tbody></table>"
	187	)
	188
	189
	190	def _blind_spots(report: CoverageReport) -> str:
	191	total = sum(len(fc.blind_spots) for fc in report.frameworks)
	192	if total == 0:
	193	return "<h2>Blind spots</h2><p>No in-scope control is left unaddressed by the corpus.</p>"
	194	parts = [
	195	"<h2>Blind spots</h2>",
	196	(
	197	f"<p>{total} in-scope control(s) are <strong>unaddressed</strong> — no finding "
	198	"in the corpus maps to them.</p>"
	199	),
	200	]
	201	for fc in report.frameworks:
	202	spots = fc.blind_spots
	203	if not spots:
	204	continue
	205	parts.append(f"<h3>{escape(fc.catalog.name)} ({len(spots)})</h3><ul>")
	206	for r in spots:
	207	fam = f" <em>· {escape(r.control.family)}</em>" if r.control.family else ""
	208	parts.append(
	209	f"<li><strong>{escape(r.control.id)}</strong> — {escape(r.control.title)}{fam}</li>"
	210	)
	211	parts.append("</ul>")
	212	return "".join(parts)
	213
	214
	215	def render(report: CoverageReport) -> str:
	216	title = report.subject or "Evidence corpus"
	217	frameworks = ", ".join(fc.catalog.framework for fc in report.frameworks)
	218	body = [
	219	"<!doctype html><html lang='en'><head><meta charset='utf-8'>",
	220	"<meta name='viewport' content='width=device-width, initial-scale=1'>",
	221	f"<title>Control Coverage — {escape(title)}</title>",
	222	f"<style>{CSS}</style></head><body><main>",
	223	f"<h1>Control Coverage — {escape(title)}</h1>",
	224	(
	225	f'<p class="meta">Generated {escape(report.generated_at)} · '
	226	f"{report.source_count} evidence source(s) · frameworks: {escape(frameworks)}</p>"
	227	),
	228	(
	229	'<p class="note">Coverage measures how much of a framework the evidence corpus '
	230	"addresses — not whether the organization is compliant. An unaddressed control is "
	231	"a gap in <em>evidence</em>, which may reflect a real control gap or simply a signal "
	232	"not yet collected. The final judgment belongs to the organization and its auditor.</p>"
	233	),
	234	"<h2>Summary</h2>",
	235	_summary_table(report),
	236	_blind_spots(report),
	237	]
	238	for fc in report.frameworks:
	239	body.append(_framework_section(fc))
	240
	241	if report.orphan_codes:
	242	codes = "".join(f"<li><code>{escape(c)}</code></li>" for c in report.orphan_codes)
	243	body.append(
	244	"<h2>Unmatched control codes</h2><p>The corpus cites these codes, but no loaded "
	245	f"catalog defines them (typos, renamed, or out-of-catalog):</p><ul>{codes}</ul>"
	246	)
	247
	248	body.append(
	249	"<footer>Generated by control-coverage · Audit Labs. Evidence, not a verdict.</footer>"
	250	)
	251	body.append("</main></body></html>")
	252	return "".join(body)

control_coverage/reporters/json.py added +55

@@ -0,0 +1,55 @@
	1	"""JSON renderer — the coverage result as a machine-readable document.
	2
	3	Stable key order so two runs diff cleanly. Suitable for dashboards, ticketing,
	4	or gating a pipeline on the coverage percentage.
	5	"""
	6
	7	from __future__ import annotations
	8
	9	import json as _json
	10	from typing import TYPE_CHECKING
	11
	12	if TYPE_CHECKING:
	13	from ..coverage import CoverageReport
	14
	15
	16	def to_dict(report: CoverageReport) -> dict:
	17	return {
	18	"subject": report.subject,
	19	"generated_at": report.generated_at,
	20	"source_count": report.source_count,
	21	"frameworks": [
	22	{
	23	"framework": fc.catalog.framework,
	24	"name": fc.catalog.name,
	25	"version": fc.catalog.version,
	26	"catalog_coverage": fc.catalog.coverage,
	27	"in_scope": fc.in_scope,
	28	"addressed": fc.addressed,
	29	"supported": fc.supported,
	30	"coverage_pct": fc.coverage_pct,
	31	"assured_pct": fc.assured_pct,
	32	"counts": fc.counts,
	33	"controls": [
	34	{
	35	"id": r.control.id,
	36	"code": r.control.code,
	37	"title": r.control.title,
	38	"family": r.control.family,
	39	"state": r.state,
	40	"owner": r.owner,
	41	"exclusion_reason": r.exclusion_reason,
	42	"checked_by": sorted({o.rule_id for o in r.observations if o.rule_id}),
	43	"sources": sorted({o.source for o in r.observations}),
	44	}
	45	for r in fc.results
	46	],
	47	}
	48	for fc in report.frameworks
	49	],
	50	"orphan_codes": report.orphan_codes,
	51	}
	52
	53
	54	def render(report: CoverageReport) -> str:
	55	return _json.dumps(to_dict(report), indent=2, sort_keys=False) + "\n"

control_coverage/reporters/markdown.py added +153

@@ -0,0 +1,153 @@
	1	"""Markdown renderer — the coverage matrix and blind-spot list, auditor-facing."""
	2
	3	from __future__ import annotations
	4
	5	from typing import TYPE_CHECKING
	6
	7	from ..coverage import (
	8	ASSERTED,
	9	FAILING,
	10	OUT_OF_SCOPE,
	11	STATE_ORDER,
	12	SUPPORTED,
	13	UNADDRESSED,
	14	)
	15
	16	if TYPE_CHECKING:
	17	from ..coverage import CoverageReport, FrameworkCoverage
	18
	19	_STATE_LABEL = {
	20	SUPPORTED: "supported",
	21	FAILING: "failing",
	22	ASSERTED: "asserted",
	23	UNADDRESSED: "unaddressed",
	24	OUT_OF_SCOPE: "out of scope",
	25	}
	26	_STATE_MARK = {
	27	SUPPORTED: "✓",
	28	FAILING: "✗",
	29	ASSERTED: "◐",
	30	UNADDRESSED: "○",
	31	OUT_OF_SCOPE: "—",
	32	}
	33
	34
	35	def _evidence_note(result) -> str:
	36	"""A short 'checked by' cell: rule ids or the exclusion reason."""
	37	if result.state == OUT_OF_SCOPE:
	38	return f"_excluded: {result.exclusion_reason}_"
	39	if not result.observations:
	40	return ""
	41	rules = sorted({o.rule_id for o in result.observations if o.rule_id})
	42	return ", ".join(f"`{r}`" for r in rules)
	43
	44
	45	def _framework_section(fc: FrameworkCoverage) -> list[str]:
	46	cat = fc.catalog
	47	counts = fc.counts
	48	out: list[str] = []
	49	out.append(f"## {cat.name}")
	50	out.append("")
	51	suffix = "" if cat.complete else " _(partial catalog — coverage is of the shipped subset)_"
	52	out.append(
	53	f"Coverage {fc.coverage_pct}% ({fc.addressed}/{fc.in_scope} in-scope "
	54	f"controls addressed) · assured {fc.assured_pct}% "
	55	f"({fc.supported} supported){suffix}"
	56	)
	57	out.append("")
	58	out.append(
	59	"\| " + " · ".join(
	60	f"{_STATE_MARK[s]} {counts[s]} {_STATE_LABEL[s]}"
	61	for s in STATE_ORDER
	62	if counts[s]
	63	) + " \|"
	64	)
	65	out.append("\|" + "---\|")
	66	out.append("")
	67	out.append("\| Control \| Status \| Description \| Checked by \|")
	68	out.append("\| --- \| --- \| --- \| --- \|")
	69	for r in fc.results:
	70	mark = _STATE_MARK[r.state]
	71	label = _STATE_LABEL[r.state]
	72	out.append(
	73	f"\| {r.control.id} \| {mark} {label} \| {r.control.title} \| {_evidence_note(r)} \|"
	74	)
	75	out.append("")
	76	return out
	77
	78
	79	def _blind_spots(report: CoverageReport) -> list[str]:
	80	out: list[str] = ["## Blind spots", ""]
	81	total = sum(len(fc.blind_spots) for fc in report.frameworks)
	82	if total == 0:
	83	out.append("_No in-scope control is left unaddressed by the corpus._")
	84	out.append("")
	85	return out
	86	out.append(
	87	f"{total} in-scope control(s) are unaddressed — no finding in the corpus "
	88	"maps to them. These are the framework requirements the evidence does not "
	89	"look at yet."
	90	)
	91	out.append("")
	92	for fc in report.frameworks:
	93	spots = fc.blind_spots
	94	if not spots:
	95	continue
	96	out.append(f"### {fc.catalog.name} ({len(spots)})")
	97	out.append("")
	98	for r in spots:
	99	fam = f" · _{r.control.family}_" if r.control.family else ""
	100	out.append(f"- {r.control.id} — {r.control.title}{fam}")
	101	out.append("")
	102	return out
	103
	104
	105	def render(report: CoverageReport) -> str:
	106	out: list[str] = []
	107	title = report.subject or "Evidence corpus"
	108	out.append(f"# Control Coverage — {title}")
	109	out.append("")
	110	out.append(f"- Generated: {report.generated_at}")
	111	out.append(f"- Corpus: {report.source_count} evidence source(s)")
	112	frameworks = ", ".join(fc.catalog.framework for fc in report.frameworks)
	113	out.append(f"- Frameworks: {frameworks}")
	114	out.append("")
	115	out.append(
	116	"> Coverage measures how much of a framework the evidence corpus addresses — "
	117	"not whether the organization is compliant. An unaddressed control is a gap "
	118	"in evidence, which may reflect a real gap in controls or simply a signal "
	119	"not yet collected. The final judgment belongs to the organization and its auditor."
	120	)
	121	out.append("")
	122
	123	# Headline table across frameworks.
	124	out.append("## Summary")
	125	out.append("")
	126	out.append("\| Framework \| In scope \| Addressed \| Supported \| Failing \| Blind spots \| Coverage \| Assured \|")
	127	out.append("\| --- \| ---: \| ---: \| ---: \| ---: \| ---: \| ---: \| ---: \|")
	128	for fc in report.frameworks:
	129	c = fc.counts
	130	out.append(
	131	f"\| {fc.catalog.name} \| {fc.in_scope} \| {fc.addressed} \| {fc.supported} "
	132	f"\| {c[FAILING]} \| {len(fc.blind_spots)} \| {fc.coverage_pct}% \| {fc.assured_pct}% \|"
	133	)
	134	out.append("")
	135
	136	out.extend(_blind_spots(report))
	137
	138	for fc in report.frameworks:
	139	out.extend(_framework_section(fc))
	140
	141	if report.orphan_codes:
	142	out.append("## Unmatched control codes")
	143	out.append("")
	144	out.append(
	145	"The corpus cites these control codes, but no loaded catalog defines them. "
	146	"They are typos, renamed controls, or controls outside the bundled catalogs:"
	147	)
	148	out.append("")
	149	for code in report.orphan_codes:
	150	out.append(f"- `{code}`")
	151	out.append("")
	152
	153	return "\n".join(out).rstrip() + "\n"

control_coverage/reporters/soa.py added +68

@@ -0,0 +1,68 @@
	1	"""Statement of Applicability renderer.
	2
	3	ISO 27001 requires a Statement of Applicability (SoA): for every Annex A control,
	4	whether it applies, why, and its implementation status. This renders exactly that
	5	from the coverage result — applicability comes from the scope file, and the
	6	implementation status is derived from the evidence corpus rather than asserted by
	7	hand, so the SoA stays honest to what the evidence actually shows.
	8	"""
	9
	10	from __future__ import annotations
	11
	12	from typing import TYPE_CHECKING
	13
	14	from ..coverage import ASSERTED, FAILING, OUT_OF_SCOPE, SUPPORTED, UNADDRESSED
	15
	16	if TYPE_CHECKING:
	17	from ..coverage import CoverageReport
	18
	19	# How each assurance state reads as an implementation status in an SoA.
	20	_IMPL_STATUS = {
	21	SUPPORTED: "Implemented — supporting evidence collected",
	22	FAILING: "Deficient — evidence shows a non-supporting state",
	23	ASSERTED: "Claimed — mapped, but evidence not yet obtained",
	24	UNADDRESSED: "Not evidenced — no evidence collected yet",
	25	OUT_OF_SCOPE: "Excluded",
	26	}
	27
	28
	29	def _justification(result) -> str:
	30	if result.state == OUT_OF_SCOPE:
	31	return result.exclusion_reason
	32	rules = sorted({o.rule_id for o in result.observations if o.rule_id})
	33	sources = sorted({o.source for o in result.observations})
	34	if rules:
	35	return f"Evidenced by {', '.join(rules)} in {', '.join(sources)}."
	36	return "No control in the evidence corpus addresses this yet."
	37
	38
	39	def render(report: CoverageReport) -> str:
	40	out: list[str] = []
	41	subject = report.subject or "the organization"
	42	out.append(f"# Statement of Applicability — {report.subject or 'Untitled'}")
	43	out.append("")
	44	out.append(f"- Generated: {report.generated_at}")
	45	out.append(f"- Derived from: {report.source_count} evidence source(s)")
	46	out.append("")
	47	out.append(
	48	f"This Statement of Applicability records, for each control in scope for "
	49	f"{subject}, whether it applies and its implementation status. Applicability "
	50	"decisions come from the documented scope; implementation status is derived "
	51	"from collected evidence, not asserted."
	52	)
	53	out.append("")
	54
	55	for fc in report.frameworks:
	56	out.append(f"## {fc.catalog.name}")
	57	out.append("")
	58	out.append("\| Control \| Description \| Applicable \| Status \| Justification \| Owner \|")
	59	out.append("\| --- \| --- \| --- \| --- \| --- \| --- \|")
	60	for r in fc.results:
	61	applicable = "No" if r.state == OUT_OF_SCOPE else "Yes"
	62	out.append(
	63	f"\| {r.control.id} \| {r.control.title} \| {applicable} "
	64	f"\| {_IMPL_STATUS[r.state]} \| {_justification(r)} \| {r.owner} \|"
	65	)
	66	out.append("")
	67
	68	return "\n".join(out).rstrip() + "\n"

control_coverage/scope.py added +106

@@ -0,0 +1,106 @@
	1	"""Scope — which controls are in play, and which are excluded with justification.
	2
	3	Not every control applies to every organization. ISO 27001 formalizes this as the
	4	Statement of Applicability (SoA): for each Annex A control, a decision to apply
	5	it or not, and the reason. This module reads a small YAML scope file expressing
	6	exactly that, so coverage is computed over in-scope controls and exclusions are
	7	recorded rather than silently counted as gaps::
	8
	9	subject: Acme Production
	10	frameworks: [SOC2, ISO]
	11	exclusions:
	12	- {control: ISO:A.5.7, reason: "No formal threat-intel program; risk accepted 2026-Q1."}
	13	- {control: ISO:A.7.1, reason: "Fully cloud-hosted; no physical premises in scope."}
	14	exclude_families:
	15	- {framework: SOC2, family: Privacy, reason: "Privacy category not in the SOC 2 audit scope."}
	16	owners:
	17	SOC2:CC6.1: platform-team
	18
	19	``exclude_families`` removes a whole category at once — a SOC 2 Trust Services
	20	category, an ISO Annex A theme, a NIST family — which is how audit scope is actually
	21	decided (a SOC 2 report covers Security and maybe Availability, rarely Privacy).
	22
	23	Every exclusion, per-control or per-family, must carry a reason — an exclusion without
	24	justification is the single most common SoA audit finding, so we reject it rather than
	25	accept it.
	26	"""
	27
	28	from __future__ import annotations
	29
	30	from dataclasses import dataclass, field
	31	from pathlib import Path
	32
	33	import yaml
	34
	35
	36	@dataclass
	37	class Scope:
	38	"""A parsed scope / Statement of Applicability."""
	39
	40	subject: str = ""
	41	frameworks: list[str] = field(default_factory=list)
	42	# control code -> justification for excluding it
	43	exclusions: dict[str, str] = field(default_factory=dict)
	44	# (framework, family) -> justification for excluding a whole family/category
	45	family_exclusions: dict[tuple[str, str], str] = field(default_factory=dict)
	46	# control code -> owning team/person (optional metadata)
	47	owners: dict[str, str] = field(default_factory=dict)
	48
	49	def excluded(self, code: str) -> bool:
	50	return code in self.exclusions
	51
	52	def reason(self, code: str) -> str:
	53	return self.exclusions.get(code, "")
	54
	55	def family_excluded(self, framework: str, family: str) -> bool:
	56	return (framework, family) in self.family_exclusions
	57
	58	def family_reason(self, framework: str, family: str) -> str:
	59	return self.family_exclusions.get((framework, family), "")
	60
	61	def owner(self, code: str) -> str:
	62	return self.owners.get(code, "")
	63
	64
	65	def empty() -> Scope:
	66	"""A scope that excludes nothing — every catalog control is in scope."""
	67	return Scope()
	68
	69
	70	def load(path: str \| Path) -> Scope:
	71	"""Load a scope file, validating that every exclusion carries a reason."""
	72	raw = yaml.safe_load(Path(path).read_text(encoding="utf-8")) or {}
	73
	74	exclusions: dict[str, str] = {}
	75	for i, item in enumerate(raw.get("exclusions", [])):
	76	if not isinstance(item, dict) or "control" not in item:
	77	raise ValueError(f"exclusion #{i + 1} must be a mapping with a 'control' key")
	78	code = str(item["control"]).strip()
	79	reason = str(item.get("reason", "")).strip()
	80	if not reason:
	81	raise ValueError(f"exclusion for '{code}' needs a non-empty 'reason'")
	82	exclusions[code] = reason
	83
	84	family_exclusions: dict[tuple[str, str], str] = {}
	85	for i, item in enumerate(raw.get("exclude_families", [])):
	86	if not isinstance(item, dict) or "framework" not in item or "family" not in item:
	87	raise ValueError(
	88	f"exclude_families #{i + 1} must be a mapping with 'framework' and 'family' keys"
	89	)
	90	framework = str(item["framework"]).strip()
	91	family = str(item["family"]).strip()
	92	reason = str(item.get("reason", "")).strip()
	93	if not reason:
	94	raise ValueError(f"family exclusion for '{framework}:{family}' needs a non-empty 'reason'")
	95	family_exclusions[(framework, family)] = reason
	96
	97	owners = {str(k): str(v) for k, v in (raw.get("owners") or {}).items()}
	98	frameworks = [str(f) for f in (raw.get("frameworks") or [])]
	99
	100	return Scope(
	101	subject=str(raw.get("subject", "")),
	102	frameworks=frameworks,
	103	exclusions=exclusions,
	104	family_exclusions=family_exclusions,
	105	owners=owners,
	106	)

control_coverage/trend.py added +332

@@ -0,0 +1,332 @@
	1	"""Coverage trend — how control coverage moved between two corpora.
	2
	3	`audit-report` diffs two evidence packages; this diffs two whole corpora at
	4	the framework-coverage level. Evaluate an earlier corpus and a current one with
	5	the same catalogs and scope, then compare each control's assurance state to see
	6	what improved, what regressed, and how the coverage percentage moved.
	7
	8	States are ranked ``supported > failing > asserted > unaddressed`` — going from
	9	"no data" to "failing data" still counts as more assurance, because you now have
	10	evidence. Two transitions are called out specially because they move the coverage
	11	numerator: gained (a blind spot became addressed) and lost (an addressed
	12	control became a blind spot).
	13	"""
	14
	15	from __future__ import annotations
	16
	17	from dataclasses import dataclass, field
	18
	19	from .coverage import (
	20	ASSERTED,
	21	FAILING,
	22	OUT_OF_SCOPE,
	23	SUPPORTED,
	24	UNADDRESSED,
	25	CoverageReport,
	26	)
	27
	28	# Assurance rank; higher is more assured. out_of_scope is handled separately.
	29	_RANK = {SUPPORTED: 3, FAILING: 2, ASSERTED: 1, UNADDRESSED: 0}
	30	_ADDRESSED = {SUPPORTED, FAILING, ASSERTED}
	31
	32	REGRESSED = "regressed"
	33	IMPROVED = "improved"
	34	GAINED = "gained"
	35	LOST = "lost"
	36	RESCOPED = "rescoped"
	37	UNCHANGED = "unchanged"
	38
	39	# Order categories appear in a report (most urgent first).
	40	CATEGORY_ORDER = [REGRESSED, LOST, GAINED, IMPROVED, RESCOPED, UNCHANGED]
	41	# Categories that count as a regression for the CI gate.
	42	_REGRESSION = {REGRESSED, LOST}
	43
	44
	45	def _categorize(old: str, new: str) -> str:
	46	if old == new:
	47	return UNCHANGED
	48	if OUT_OF_SCOPE in (old, new):
	49	return RESCOPED
	50	if old == UNADDRESSED and new in _ADDRESSED:
	51	return GAINED
	52	if old in _ADDRESSED and new == UNADDRESSED:
	53	return LOST
	54	return IMPROVED if _RANK[new] > _RANK[old] else REGRESSED
	55
	56
	57	@dataclass
	58	class ControlDelta:
	59	"""How one control's assurance state changed between the two corpora."""
	60
	61	framework: str
	62	id: str
	63	title: str
	64	old_state: str
	65	new_state: str
	66	category: str
	67
	68
	69	@dataclass
	70	class FrameworkTrend:
	71	"""Coverage movement for one framework."""
	72
	73	framework: str
	74	name: str
	75	deltas: list[ControlDelta]
	76	old_coverage_pct: float
	77	new_coverage_pct: float
	78
	79	def by_category(self, category: str) -> list[ControlDelta]:
	80	return [d for d in self.deltas if d.category == category]
	81
	82	@property
	83	def counts(self) -> dict[str, int]:
	84	counts = {c: 0 for c in CATEGORY_ORDER}
	85	for d in self.deltas:
	86	counts[d.category] += 1
	87	return counts
	88
	89	@property
	90	def coverage_delta(self) -> float:
	91	return round(self.new_coverage_pct - self.old_coverage_pct, 1)
	92
	93	@property
	94	def regressions(self) -> int:
	95	return sum(1 for d in self.deltas if d.category in _REGRESSION)
	96
	97
	98	@dataclass
	99	class TrendReport:
	100	subject: str
	101	generated_at: str
	102	frameworks: list[FrameworkTrend] = field(default_factory=list)
	103
	104	@property
	105	def total_regressions(self) -> int:
	106	return sum(fc.regressions for fc in self.frameworks)
	107
	108
	109	def compare(baseline: CoverageReport, current: CoverageReport) -> TrendReport:
	110	"""Diff two coverage reports evaluated with the same catalogs and scope."""
	111	old_by_fw = {fc.catalog.framework: fc for fc in baseline.frameworks}
	112
	113	frameworks: list[FrameworkTrend] = []
	114	for cur in current.frameworks:
	115	base = old_by_fw.get(cur.catalog.framework)
	116	if base is None:
	117	continue # framework only appears in the current run
	118	old_state = {r.control.id: r.state for r in base.results}
	119	deltas: list[ControlDelta] = []
	120	for r in cur.results:
	121	prev = old_state.get(r.control.id, UNADDRESSED)
	122	deltas.append(
	123	ControlDelta(
	124	framework=cur.catalog.framework,
	125	id=r.control.id,
	126	title=r.control.title,
	127	old_state=prev,
	128	new_state=r.state,
	129	category=_categorize(prev, r.state),
	130	)
	131	)
	132	frameworks.append(
	133	FrameworkTrend(
	134	framework=cur.catalog.framework,
	135	name=cur.catalog.name,
	136	deltas=deltas,
	137	old_coverage_pct=base.coverage_pct,
	138	new_coverage_pct=cur.coverage_pct,
	139	)
	140	)
	141
	142	return TrendReport(
	143	subject=current.subject,
	144	generated_at=current.generated_at,
	145	frameworks=frameworks,
	146	)
	147
	148
	149	# --- renderers -------------------------------------------------------------
	150
	151	_ARROW = {
	152	REGRESSED: "▼",
	153	LOST: "▼",
	154	GAINED: "▲",
	155	IMPROVED: "▲",
	156	RESCOPED: "◆",
	157	UNCHANGED: "·",
	158	}
	159
	160
	161	def to_dict(report: TrendReport) -> dict:
	162	return {
	163	"subject": report.subject,
	164	"generated_at": report.generated_at,
	165	"total_regressions": report.total_regressions,
	166	"frameworks": [
	167	{
	168	"framework": fc.framework,
	169	"name": fc.name,
	170	"old_coverage_pct": fc.old_coverage_pct,
	171	"new_coverage_pct": fc.new_coverage_pct,
	172	"coverage_delta": fc.coverage_delta,
	173	"counts": fc.counts,
	174	"changes": [
	175	{
	176	"id": d.id,
	177	"title": d.title,
	178	"old_state": d.old_state,
	179	"new_state": d.new_state,
	180	"category": d.category,
	181	}
	182	for d in fc.deltas
	183	if d.category != UNCHANGED
	184	],
	185	}
	186	for fc in report.frameworks
	187	],
	188	}
	189
	190
	191	def render_json(report: TrendReport) -> str:
	192	import json
	193
	194	return json.dumps(to_dict(report), indent=2, sort_keys=False) + "\n"
	195
	196
	197	def render_markdown(report: TrendReport) -> str:
	198	out: list[str] = []
	199	out.append(f"# Coverage Trend — {report.subject or 'Evidence corpus'}")
	200	out.append("")
	201	out.append(f"- Generated: {report.generated_at}")
	202	out.append(f"- Regressions: {report.total_regressions}")
	203	out.append("")
	204
	205	out.append("\| Framework \| Coverage (was → now) \| Δ \| Regressed \| Lost \| Gained \| Improved \|")
	206	out.append("\| --- \| --- \| ---: \| ---: \| ---: \| ---: \| ---: \|")
	207	for fc in report.frameworks:
	208	c = fc.counts
	209	sign = "+" if fc.coverage_delta >= 0 else ""
	210	out.append(
	211	f"\| {fc.name} \| {fc.old_coverage_pct}% → {fc.new_coverage_pct}% "
	212	f"\| {sign}{fc.coverage_delta} \| {c[REGRESSED]} \| {c[LOST]} \| {c[GAINED]} \| {c[IMPROVED]} \|"
	213	)
	214	out.append("")
	215
	216	for fc in report.frameworks:
	217	changes = [d for d in fc.deltas if d.category != UNCHANGED]
	218	if not changes:
	219	continue
	220	out.append(f"## {fc.name}")
	221	out.append("")
	222	out.append("\| Control \| Change \| Was → Now \| Description \|")
	223	out.append("\| --- \| --- \| --- \| --- \|")
	224	ordered = sorted(changes, key=lambda d: CATEGORY_ORDER.index(d.category))
	225	for d in ordered:
	226	arrow = _ARROW[d.category]
	227	out.append(
	228	f"\| {d.id} \| {arrow} {d.category} \| {d.old_state} → {d.new_state} \| {d.title} \|"
	229	)
	230	out.append("")
	231
	232	if all(all(x.category == UNCHANGED for x in fc.deltas) for fc in report.frameworks):
	233	out.append("_No control changed state between the two corpora._")
	234	out.append("")
	235
	236	return "\n".join(out).rstrip() + "\n"
	237
	238
	239	# Category badge colours, layered onto the shared coverage CSS.
	240	_TREND_CSS = """
	241	.badge.improved { background: #e5f6ea; color: #1a7f37; }
	242	.badge.gained { background: #e3eefb; color: #1667a8; }
	243	.badge.regressed { background: #fdeaea; color: #c1272d; }
	244	.badge.lost { background: #fbe7d8; color: #a8480a; }
	245	.badge.rescoped { background: #eee; color: #666; }
	246	.delta-up { color: #1a7f37; font-weight: 700; }
	247	.delta-down { color: #c1272d; font-weight: 700; }
	248	@media (prefers-color-scheme: dark) {
	249	.badge.improved { background: #12321d; color: #4ac36a; }
	250	.badge.gained { background: #12263a; color: #5aa6e6; }
	251	.badge.regressed { background: #3a1416; color: #ff6b70; }
	252	.badge.lost { background: #33220f; color: #e0913c; }
	253	.badge.rescoped { background: #26272b; color: #999; }
	254	}
	255	"""
	256
	257
	258	def render_html(report: TrendReport) -> str:
	259	from html import escape
	260
	261	from .reporters.html import CSS
	262
	263	def badge(category: str) -> str:
	264	return f'<span class="badge {category}">{_ARROW[category]} {category}</span>'
	265
	266	def delta(value: float) -> str:
	267	cls = "delta-up" if value >= 0 else "delta-down"
	268	sign = "+" if value >= 0 else ""
	269	return f'<span class="{cls}">{sign}{value}</span>'
	270
	271	title = report.subject or "Evidence corpus"
	272	body = [
	273	"<!doctype html><html lang='en'><head><meta charset='utf-8'>",
	274	"<meta name='viewport' content='width=device-width, initial-scale=1'>",
	275	f"<title>Coverage Trend — {escape(title)}</title>",
	276	f"<style>{CSS}{_TREND_CSS}</style></head><body><main>",
	277	f"<h1>Coverage Trend — {escape(title)}</h1>",
	278	(
	279	f'<p class="meta">Generated {escape(report.generated_at)} · '
	280	f"{report.total_regressions} regression(s)</p>"
	281	),
	282	"<h2>Summary</h2>",
	283	(
	284	"<table><thead><tr><th>Framework</th><th>Coverage (was → now)</th>"
	285	"<th class='num'>Δ</th><th class='num'>Regressed</th><th class='num'>Lost</th>"
	286	"<th class='num'>Gained</th><th class='num'>Improved</th></tr></thead><tbody>"
	287	),
	288	]
	289	for fc in report.frameworks:
	290	c = fc.counts
	291	body.append(
	292	"<tr>"
	293	f"<td>{escape(fc.name)}</td>"
	294	f"<td>{fc.old_coverage_pct}% → {fc.new_coverage_pct}%</td>"
	295	f'<td class="num">{delta(fc.coverage_delta)}</td>'
	296	f'<td class="num">{c[REGRESSED]}</td><td class="num">{c[LOST]}</td>'
	297	f'<td class="num">{c[GAINED]}</td><td class="num">{c[IMPROVED]}</td>'
	298	"</tr>"
	299	)
	300	body.append("</tbody></table>")
	301
	302	for fc in report.frameworks:
	303	changes = sorted(
	304	(d for d in fc.deltas if d.category != UNCHANGED),
	305	key=lambda d: CATEGORY_ORDER.index(d.category),
	306	)
	307	if not changes:
	308	continue
	309	body.append(f"<h2>{escape(fc.name)}</h2>")
	310	body.append(
	311	"<table><thead><tr><th>Control</th><th>Change</th><th>Was → Now</th>"
	312	"<th>Description</th></tr></thead><tbody>"
	313	)
	314	for d in changes:
	315	body.append(
	316	"<tr>"
	317	f"<td><strong>{escape(d.id)}</strong></td>"
	318	f"<td>{badge(d.category)}</td>"
	319	f"<td>{d.old_state} → {d.new_state}</td>"
	320	f"<td>{escape(d.title)}</td>"
	321	"</tr>"
	322	)
	323	body.append("</tbody></table>")
	324
	325	if not any(any(x.category != UNCHANGED for x in fc.deltas) for fc in report.frameworks):
	326	body.append("<p>No control changed state between the two corpora.</p>")
	327
	328	body.append(
	329	"<footer>Generated by control-coverage · Audit Labs. Evidence, not a verdict.</footer>"
	330	)
	331	body.append("</main></body></html>")
	332	return "".join(body)

examples/github-actions-coverage.yml added +43

@@ -0,0 +1,43 @@
	1	# Gate a pipeline on framework coverage.
	2	#
	3	# Assumes an earlier job produced audit-report JSON packages under ./reports/
	4	# (one per platform). This job fails the build if SOC 2 or ISO coverage drops
	5	# below the threshold, and publishes the coverage report + Statement of
	6	# Applicability as build artifacts.
	7	name: control-coverage
	8
	9	on:
	10	workflow_dispatch:
	11	schedule:
	12	- cron: "0 6 * * 1" # Mondays, 06:00 UTC
	13
	14	jobs:
	15	coverage:
	16	runs-on: ubuntu-latest
	17	steps:
	18	- uses: actions/checkout@v4
	19
	20	- uses: actions/setup-python@v5
	21	with:
	22	python-version: "3.12"
	23
	24	- name: Install control-coverage
	25	run: pip install git+https://github.com/audit-labs/control-coverage
	26
	27	# Your own step(s) should populate ./reports/*.json with audit-report output.
	28
	29	- name: Coverage report + SoA
	30	run: \|
	31	control-coverage ./reports/ \
	32	--scope examples/soa.yaml \
	33	--format md,html,json,soa \
	34	--out coverage-out/
	35
	36	- name: Fail if coverage regresses
	37	run: control-coverage ./reports/ --scope examples/soa.yaml --fail-under 60
	38
	39	- uses: actions/upload-artifact@v4
	40	if: always()
	41	with:
	42	name: coverage
	43	path: coverage-out/

examples/soa.yaml added +27

@@ -0,0 +1,27 @@
	1	# Example scope / Statement of Applicability.
	2	#
	3	# `frameworks` selects which catalogs to evaluate. Each exclusion removes a
	4	# control from the in-scope denominator and MUST carry a justification. `owners`
	5	# is optional metadata that flows through to the SoA.
	6	subject: Acme Production
	7	frameworks: [SOC2, ISO]
	8
	9	exclusions:
	10	- control: ISO:A.7.1
	11	reason: "Fully cloud-hosted; no physical premises are in scope for the ISMS."
	12	- control: ISO:A.7.2
	13	reason: "No physical premises — physical entry controls are not applicable."
	14	- control: ISO:A.5.7
	15	reason: "No formal threat-intelligence program; risk accepted by the CISO for 2026."
	16
	17	# Drop whole categories/families at once. A SOC 2 report here covers Security and
	18	# Availability only, so the other three Trust Services categories are out of scope.
	19	exclude_families:
	20	- {framework: SOC2, family: Confidentiality, reason: "Confidentiality category not in the SOC 2 audit scope."}
	21	- {framework: SOC2, family: Processing Integrity, reason: "Processing Integrity category not in the SOC 2 audit scope."}
	22	- {framework: SOC2, family: Privacy, reason: "Privacy category not in the SOC 2 audit scope."}
	23
	24	owners:
	25	SOC2:CC6.1: platform-team
	26	SOC2:CC7.2: security-ops
	27	ISO:A.5.17: identity-team

pyproject.toml added +31

@@ -0,0 +1,31 @@
	1	[build-system]
	2	requires = ["setuptools>=68"]
	3	build-backend = "setuptools.build_meta"
	4
	5	[project]
	6	name = "control-coverage"
	7	version = "0.1.0"
	8	description = "Control-first coverage and blind-spot analysis over an evidence corpus, with a Statement of Applicability."
	9	readme = "README.md"
	10	requires-python = ">=3.10"
	11	license = { text = "GPL-3.0-or-later" }
	12	authors = [{ name = "Audit Labs" }]
	13	dependencies = ["PyYAML>=6.0"]
	14
	15	[project.optional-dependencies]
	16	dev = ["pytest>=8.0", "ruff>=0.5"]
	17
	18	[project.scripts]
	19	control-coverage = "control_coverage.cli:main"
	20
	21	[project.urls]
	22	Homepage = "https://audit-labs.dev"
	23	Repository = "https://github.com/audit-labs/control-coverage"
	24
	25	[tool.setuptools]
	26	packages = ["control_coverage", "control_coverage.reporters"]
	27	include-package-data = true
	28
	29	[tool.setuptools.package-data]
	30	# Ship the bundled framework catalogs inside the wheel so the CLI works after install.
	31	"control_coverage" = ["catalogs/*.yaml"]

requirements-dev.txt added +3

@@ -0,0 +1,3 @@
	1	PyYAML>=6.0
	2	pytest>=8.0
	3	ruff>=0.5

requirements.txt added +1

	@@ -0,0 +1 @@
		1	PyYAML>=6.0

ruff.toml added +14

@@ -0,0 +1,14 @@
	1	# Ruff configuration for audit-tools.
	2	#
	3	# A few lint rules are disabled because they flag patterns this project uses
	4	# deliberately:
	5	#
	6	# BLE001 - The audit collectors and their CLI wrappers intentionally catch
	7	# broad exceptions so that one failing check never aborts a whole
	8	# audit run. The error is reported and collection continues.
	9	# DTZ011 - date.today() is used to build human-facing, date-stamped output
	10	# folder names, where the local date is the intended value.
	11	# S112 - try/except/continue is used to skip resources that are unavailable
	12	# during collection (e.g. a repo without the requested branch).
	13	[lint]
	14	ignore = ["BLE001", "DTZ011", "S112"]

tests/fixtures/aws_audit_acme_2026-01-01.json added +46

@@ -0,0 +1,46 @@
	1	{
	2	"subject": "acme",
	3	"platform": "aws",
	4	"source_package": "aws_audit_acme_2026-01-01",
	5	"generated_at": "2026-01-01 00:00:00 UTC",
	6	"summary": {"pass": 2, "fail": 1, "not_applicable": 0},
	7	"coverage": ["SOC2:CC6.1", "SOC2:CC6.6", "SOC2:CC7.2", "ISO:A.8.15"],
	8	"findings": [
	9	{
	10	"id": "aws.iam.root-mfa",
	11	"title": "Root account has MFA enabled",
	12	"status": "pass",
	13	"severity": "high",
	14	"controls": ["SOC2:CC6.1", "ISO:A.5.17", "NIST:IA-2"],
	15	"reason": "1 row asserted true",
	16	"evidence": []
	17	},
	18	{
	19	"id": "aws.ec2.no-open-sg",
	20	"title": "No security group open to 0.0.0.0/0 on admin ports",
	21	"status": "fail",
	22	"severity": "high",
	23	"controls": ["SOC2:CC6.6", "ISO:A.8.20", "NIST:SC-7"],
	24	"reason": "2 rows failed the check",
	25	"evidence": [{"group_id": "sg-1", "port": "22"}, {"group_id": "sg-2", "port": "3389"}]
	26	},
	27	{
	28	"id": "aws.cloudtrail.enabled",
	29	"title": "CloudTrail logging is enabled in all regions",
	30	"status": "pass",
	31	"severity": "high",
	32	"controls": ["SOC2:CC7.2", "ISO:A.8.15", "NIST:AU-2"],
	33	"reason": "1 row asserted true",
	34	"evidence": []
	35	},
	36	{
	37	"id": "aws.legacy.old-code",
	38	"title": "Legacy control citing an unknown code",
	39	"status": "pass",
	40	"severity": "low",
	41	"controls": ["SOC2:CC6.99"],
	42	"reason": "for orphan-code testing",
	43	"evidence": []
	44	}
	45	]
	46	}

tests/fixtures/baseline_github.json added +46

@@ -0,0 +1,46 @@
	1	{
	2	"subject": "acme",
	3	"platform": "github",
	4	"source_package": "github_audit_acme_2025-10-01",
	5	"generated_at": "2025-10-01 00:00:00 UTC",
	6	"summary": {"pass": 2, "fail": 2, "not_applicable": 0},
	7	"coverage": ["SOC2:CC6.1", "SOC2:CC6.3", "SOC2:CC7.1", "SOC2:CC9.2"],
	8	"findings": [
	9	{
	10	"id": "github.org.require-2fa",
	11	"title": "Organization requires two-factor authentication",
	12	"status": "fail",
	13	"severity": "high",
	14	"controls": ["SOC2:CC6.1", "ISO:A.5.17", "NIST:IA-2"],
	15	"reason": "2fa not enforced at the time of this snapshot",
	16	"evidence": [{"two_factor_required": "false"}]
	17	},
	18	{
	19	"id": "github.org.default-permission",
	20	"title": "Base repository permission is read or less",
	21	"status": "fail",
	22	"severity": "medium",
	23	"controls": ["SOC2:CC6.3", "ISO:A.5.15", "NIST:AC-6"],
	24	"reason": "base permission is write",
	25	"evidence": [{"default_repo_permission": "write"}]
	26	},
	27	{
	28	"id": "github.org.secret-scanning",
	29	"title": "Secret scanning push protection is on for new repos",
	30	"status": "pass",
	31	"severity": "medium",
	32	"controls": ["SOC2:CC7.1", "ISO:A.5.17", "NIST:CM-6"],
	33	"reason": "1 row asserted true",
	34	"evidence": []
	35	},
	36	{
	37	"id": "github.org.vendor-review",
	38	"title": "Third-party OAuth app access is restricted",
	39	"status": "pass",
	40	"severity": "medium",
	41	"controls": ["SOC2:CC9.2"],
	42	"reason": "1 row asserted true",
	43	"evidence": []
	44	}
	45	]
	46	}

tests/fixtures/github_audit_acme_2026-01-01.json added +46

@@ -0,0 +1,46 @@
	1	{
	2	"subject": "acme",
	3	"platform": "github",
	4	"source_package": "github_audit_acme_2026-01-01",
	5	"generated_at": "2026-01-01 00:00:00 UTC",
	6	"summary": {"pass": 2, "fail": 1, "not_applicable": 1},
	7	"coverage": ["SOC2:CC6.1", "SOC2:CC6.3", "SOC2:CC7.1", "ISO:A.5.17"],
	8	"findings": [
	9	{
	10	"id": "github.org.require-2fa",
	11	"title": "Organization requires two-factor authentication",
	12	"status": "pass",
	13	"severity": "high",
	14	"controls": ["SOC2:CC6.1", "ISO:A.5.17", "NIST:IA-2"],
	15	"reason": "1 row asserted true",
	16	"evidence": []
	17	},
	18	{
	19	"id": "github.org.default-permission",
	20	"title": "Base repository permission is read or less",
	21	"status": "fail",
	22	"severity": "medium",
	23	"controls": ["SOC2:CC6.3", "ISO:A.5.15", "NIST:AC-6"],
	24	"reason": "base permission is write",
	25	"evidence": [{"default_repo_permission": "write"}]
	26	},
	27	{
	28	"id": "github.org.secret-scanning",
	29	"title": "Secret scanning push protection is on for new repos",
	30	"status": "pass",
	31	"severity": "medium",
	32	"controls": ["SOC2:CC7.1", "ISO:A.5.17", "NIST:CM-6"],
	33	"reason": "1 row asserted true",
	34	"evidence": []
	35	},
	36	{
	37	"id": "github.branch.require-reviews",
	38	"title": "Default branch requires pull request reviews",
	39	"status": "not_applicable",
	40	"severity": "high",
	41	"controls": ["SOC2:CC8.1", "ISO:A.8.32", "NIST:CM-3"],
	42	"reason": "table 'branch_protections' not in package",
	43	"evidence": []
	44	}
	45	]
	46	}

tests/fixtures/scope.yaml added +11

@@ -0,0 +1,11 @@
	1	# Example scope / Statement of Applicability for the test corpus.
	2	subject: Acme Production
	3	frameworks: [SOC2, ISO]
	4	exclusions:
	5	- control: ISO:A.7.1
	6	reason: "Fully cloud-hosted; no physical premises are in scope for the ISMS."
	7	- control: ISO:A.5.7
	8	reason: "No formal threat-intelligence program; risk accepted by the CISO for 2026."
	9	owners:
	10	SOC2:CC6.1: platform-team
	11	ISO:A.5.17: identity-team

tests/test_catalog.py added +74

@@ -0,0 +1,74 @@
	1	"""Tests for loading framework catalogs."""
	2
	3	import pytest
	4
	5	from control_coverage import catalog
	6
	7
	8	def test_soc2_is_complete_common_criteria():
	9	cat = catalog.load("soc2")
	10	assert cat.framework == "SOC2"
	11	assert cat.complete
	12	ids = {c.id for c in cat.controls}
	13	# A representative spread across every Common Criteria group.
	14	for cc in ["CC1.1", "CC5.3", "CC6.8", "CC7.5", "CC8.1", "CC9.2"]:
	15	assert cc in ids
	16
	17
	18	def test_soc2_has_all_five_tsc_categories():
	19	cat = catalog.load("soc2")
	20	families = {c.family for c in cat.controls}
	21	assert {"Availability", "Confidentiality", "Processing Integrity", "Privacy"} <= families
	22	ids = {c.id for c in cat.controls}
	23	for cid in ["C1.1", "PI1.5", "P6.7", "P8.1"]:
	24	assert cid in ids
	25	assert len(cat.controls) == 61
	26
	27
	28	def test_iso_has_all_93_annex_a_controls():
	29	cat = catalog.load("iso")
	30	assert cat.framework == "ISO"
	31	assert cat.complete
	32	assert len(cat.controls) == 93
	33
	34
	35	def test_nist_is_the_moderate_baseline():
	36	cat = catalog.load("nist")
	37	assert cat.complete # complete relative to the moderate baseline
	38	assert "Moderate" in cat.name
	39	assert len(cat.controls) > 150
	40	ids = {c.id for c in cat.controls}
	41	# A spread across families the ecosystem's rulesets cite and beyond.
	42	for cid in ["AC-6", "AU-2", "CM-6", "IA-2", "SC-7", "SI-2", "SR-3"]:
	43	assert cid in ids
	44
	45
	46	def test_control_code_joins_framework_and_id():
	47	cat = catalog.load("soc2")
	48	ctrl = next(c for c in cat.controls if c.id == "CC6.1")
	49	assert ctrl.code == "SOC2:CC6.1"
	50
	51
	52	def test_titles_with_commas_survive_parsing():
	53	cat = catalog.load("soc2")
	54	ctrl = next(c for c in cat.controls if c.id == "CC1.3")
	55	assert "reporting lines" in ctrl.title # flow-scalar comma bug regression
	56
	57
	58	def test_aliases_resolve():
	59	assert catalog.load("iso 27001").framework == "ISO"
	60	assert catalog.load("800-53").framework == "NIST"
	61
	62
	63	def test_unknown_framework_raises():
	64	with pytest.raises(ValueError, match="unknown framework"):
	65	catalog.load("hipaa")
	66
	67
	68	def test_load_frameworks_dedupes_and_sorts():
	69	cats = catalog.load_frameworks(["NIST", "SOC2", "soc 2"])
	70	assert [c.framework for c in cats] == ["NIST", "SOC2"]
	71
	72
	73	def test_available_lists_bundled():
	74	assert set(catalog.available()) == {"SOC2", "ISO", "NIST"}

tests/test_cli.py added +117

@@ -0,0 +1,117 @@
	1	"""Tests for the command-line interface."""
	2
	3	from pathlib import Path
	4
	5	import pytest
	6
	7	from control_coverage import cli
	8
	9	FIXTURES = Path(__file__).parent / "fixtures"
	10	GITHUB = str(FIXTURES / "github_audit_acme_2026-01-01.json")
	11	AWS = str(FIXTURES / "aws_audit_acme_2026-01-01.json")
	12	SCOPE = str(FIXTURES / "scope.yaml")
	13
	14
	15	def test_stdout_markdown_default(capsys):
	16	rc = cli.main([GITHUB, AWS, "--framework", "SOC2"])
	17	out = capsys.readouterr().out
	18	assert rc == 0
	19	assert "# Control Coverage" in out
	20	assert "Coverage" in out
	21
	22
	23	def test_frameworks_inferred_from_corpus(capsys):
	24	cli.main([GITHUB, "--format", "json"])
	25	out = capsys.readouterr().out
	26	# github fixture cites SOC2, ISO, NIST codes -> all three inferred.
	27	for fw in ("SOC2", "ISO", "NIST"):
	28	assert f'"framework": "{fw}"' in out
	29
	30
	31	def test_scope_file_supplies_frameworks_and_subject(capsys):
	32	cli.main([GITHUB, AWS, "--scope", SCOPE, "--format", "json"])
	33	out = capsys.readouterr().out
	34	assert '"subject": "Acme Production"' in out
	35	assert '"framework": "NIST"' not in out # scope lists only SOC2, ISO
	36
	37
	38	def test_blind_spots_mode(capsys):
	39	rc = cli.main([GITHUB, "--framework", "SOC2", "--blind-spots"])
	40	out = capsys.readouterr().out
	41	assert rc == 0
	42	assert "unaddressed" in out
	43	assert "SOC2:CC1.1" in out
	44
	45
	46	def test_fail_under_gate_trips(capsys):
	47	rc = cli.main([GITHUB, "--framework", "SOC2", "--fail-under", "90"])
	48	assert rc == 1
	49	err = capsys.readouterr().err
	50	assert "coverage gate" in err
	51
	52
	53	def test_fail_under_gate_passes(capsys):
	54	rc = cli.main([GITHUB, "--framework", "SOC2", "--fail-under", "1"])
	55	assert rc == 0
	56
	57
	58	def test_out_dir_writes_files(tmp_path, capsys):
	59	rc = cli.main(
	60	[GITHUB, "--scope", SCOPE, "--format", "md,html,json,soa", "--out", str(tmp_path)]
	61	)
	62	assert rc == 0
	63	written = {p.name for p in tmp_path.iterdir()}
	64	assert "soa.md" in written
	65	assert any(n.endswith(".html") for n in written)
	66	assert any(n.endswith(".json") for n in written)
	67
	68
	69	def test_missing_reports_errors():
	70	with pytest.raises(SystemExit):
	71	cli.main([str(FIXTURES / "nope.json"), "--framework", "SOC2"])
	72
	73
	74	BASELINE = str(FIXTURES / "baseline_github.json")
	75
	76
	77	def test_trend_mode_markdown(capsys):
	78	rc = cli.main([GITHUB, AWS, "--framework", "SOC2", "--baseline", BASELINE])
	79	out = capsys.readouterr().out
	80	assert rc == 0
	81	assert "# Coverage Trend" in out
	82
	83
	84	def test_trend_html_output(tmp_path):
	85	cli.main([GITHUB, AWS, "--framework", "SOC2", "--baseline", BASELINE,
	86	"--format", "html,json", "--out", str(tmp_path)])
	87	names = {p.name for p in tmp_path.iterdir()}
	88	assert "trend.html" in names and "trend.json" in names
	89
	90
	91	def test_crosswalk_mode(capsys):
	92	rc = cli.main([GITHUB, AWS, "--framework", "SOC2,ISO", "--crosswalk"])
	93	out = capsys.readouterr().out
	94	assert rc == 0
	95	assert "Minimal evidence set" in out
	96
	97
	98	def test_crosswalk_and_baseline_conflict():
	99	with pytest.raises(SystemExit, match="cannot be combined"):
	100	cli.main([GITHUB, "--crosswalk", "--baseline", BASELINE])
	101
	102
	103	def test_trend_rejects_soa_format():
	104	with pytest.raises(SystemExit, match="trend mode supports"):
	105	cli.main([GITHUB, "--framework", "SOC2", "--baseline", BASELINE, "--format", "soa"])
	106
	107
	108	def test_family_exclusion_via_cli(tmp_path, capsys):
	109	scope_file = tmp_path / "scope.yaml"
	110	scope_file.write_text(
	111	"frameworks: [SOC2]\n"
	112	"exclude_families:\n"
	113	" - {framework: SOC2, family: Privacy, reason: 'Not in scope.'}\n"
	114	)
	115	cli.main([GITHUB, "--scope", str(scope_file), "--format", "json"])
	116	out = capsys.readouterr().out
	117	assert '"state": "out_of_scope"' in out

tests/test_corpus.py added +42

@@ -0,0 +1,42 @@
	1	"""Tests for loading an evidence corpus from audit-report JSON."""
	2
	3	from pathlib import Path
	4
	5	import pytest
	6
	7	from control_coverage import corpus
	8
	9	FIXTURES = Path(__file__).parent / "fixtures"
	10	GITHUB = FIXTURES / "github_audit_acme_2026-01-01.json"
	11	AWS = FIXTURES / "aws_audit_acme_2026-01-01.json"
	12
	13
	14	def test_one_observation_per_finding_control_pair():
	15	obs = corpus.load_report(GITHUB)
	16	# 4 findings with 3+3+3+3 controls = 12 observations.
	17	assert len(obs) == 12
	18
	19
	20	def test_observation_carries_provenance():
	21	obs = corpus.load_report(GITHUB)
	22	o = next(o for o in obs if o.control == "SOC2:CC6.3")
	23	assert o.status == corpus.FAIL
	24	assert o.rule_id == "github.org.default-permission"
	25	assert o.source == "github_audit_acme_2026-01-01"
	26
	27
	28	def test_load_corpus_flattens_multiple_reports():
	29	obs = corpus.load_corpus([GITHUB, AWS])
	30	sources = {o.source for o in obs}
	31	assert sources == {"github_audit_acme_2026-01-01", "aws_audit_acme_2026-01-01"}
	32
	33
	34	def test_directory_is_expanded_to_json_files():
	35	obs = corpus.load_corpus([FIXTURES])
	36	assert len(obs) > 0
	37	assert any(o.source.startswith("aws_") for o in obs)
	38
	39
	40	def test_empty_directory_raises(tmp_path):
	41	with pytest.raises(ValueError, match="no audit-report JSON"):
	42	corpus.load_corpus([tmp_path])

tests/test_coverage.py added +91

@@ -0,0 +1,91 @@
	1	"""Tests for the coverage engine — the heart of the tool."""
	2
	3	from pathlib import Path
	4
	5	from control_coverage import catalog, corpus, scope
	6	from control_coverage.coverage import (
	7	ASSERTED,
	8	FAILING,
	9	OUT_OF_SCOPE,
	10	SUPPORTED,
	11	UNADDRESSED,
	12	evaluate,
	13	)
	14
	15	FIXTURES = Path(__file__).parent / "fixtures"
	16	GITHUB = FIXTURES / "github_audit_acme_2026-01-01.json"
	17	AWS = FIXTURES / "aws_audit_acme_2026-01-01.json"
	18
	19
	20	def _state(fc, control_id):
	21	return next(r.state for r in fc.results if r.control.id == control_id)
	22
	23
	24	def _report(frameworks, scp=None):
	25	obs = corpus.load_corpus([GITHUB, AWS])
	26	cats = catalog.load_frameworks(frameworks)
	27	return evaluate(cats, obs, scope=scp)
	28
	29
	30	def test_worst_wins_and_blind_spots_for_soc2():
	31	fc = _report(["SOC2"]).frameworks[0]
	32	assert _state(fc, "CC6.1") == SUPPORTED # two passes across github + aws
	33	assert _state(fc, "CC6.3") == FAILING # one fail
	34	assert _state(fc, "CC6.6") == FAILING
	35	assert _state(fc, "CC8.1") == ASSERTED # only not_applicable observations
	36	assert _state(fc, "CC1.1") == UNADDRESSED # nothing maps here
	37
	38
	39	def test_soc2_rollup_numbers():
	40	fc = _report(["SOC2"]).frameworks[0]
	41	assert fc.in_scope == 61 # full five-category Trust Services Criteria
	42	assert fc.supported == 3
	43	assert fc.counts[FAILING] == 2
	44	assert fc.counts[ASSERTED] == 1
	45	assert fc.addressed == 6
	46	assert len(fc.blind_spots) == 55
	47	assert fc.coverage_pct == 9.8 # 6 / 61
	48	assert fc.assured_pct == 4.9 # 3 / 61
	49
	50
	51	def test_scope_marks_controls_out_of_scope_with_reason():
	52	scp = scope.load(FIXTURES / "scope.yaml")
	53	fc = next(f for f in _report(["ISO"], scp).frameworks if f.catalog.framework == "ISO")
	54	assert _state(fc, "A.7.1") == OUT_OF_SCOPE
	55	assert _state(fc, "A.5.7") == OUT_OF_SCOPE
	56	assert fc.in_scope == 91 # 93 Annex A controls minus 2 exclusions
	57	excluded = next(r for r in fc.results if r.control.id == "A.7.1")
	58	assert "cloud-hosted" in excluded.exclusion_reason
	59
	60
	61	def test_orphan_only_flags_loaded_frameworks():
	62	# CC6.99 is a SOC2 typo; NIST codes are cited but NIST is not loaded here.
	63	report = _report(["SOC2", "ISO"])
	64	assert "SOC2:CC6.99" in report.orphan_codes
	65	assert not any(c.startswith("NIST:") for c in report.orphan_codes)
	66
	67
	68	def test_family_exclusion_marks_whole_category_out_of_scope():
	69	from control_coverage.scope import Scope
	70
	71	scp = Scope(family_exclusions={("SOC2", "Privacy"): "Not in the SOC 2 audit scope."})
	72	fc = _report(["SOC2"], scp).frameworks[0]
	73	privacy = [r for r in fc.results if r.control.family == "Privacy"]
	74	assert privacy # the catalog has Privacy controls
	75	assert all(r.state == OUT_OF_SCOPE for r in privacy)
	76	assert all("audit scope" in r.exclusion_reason for r in privacy)
	77	# Security (Common Criteria) controls remain in scope.
	78	cc61 = next(r for r in fc.results if r.control.id == "CC6.1")
	79	assert cc61.state != OUT_OF_SCOPE
	80
	81
	82	def test_owner_is_attached_from_scope():
	83	scp = scope.load(FIXTURES / "scope.yaml")
	84	fc = _report(["SOC2"], scp).frameworks[0]
	85	owner = next(r.owner for r in fc.results if r.control.id == "CC6.1")
	86	assert owner == "platform-team"
	87
	88
	89	def test_source_count_reflects_distinct_packages():
	90	report = _report(["SOC2"])
	91	assert report.source_count == 2

tests/test_crosswalk.py added +67

@@ -0,0 +1,67 @@
	1	"""Tests for the evidence crosswalk and minimal-evidence set."""
	2
	3	import json
	4	from pathlib import Path
	5
	6	from control_coverage import catalog, corpus, crosswalk
	7	from control_coverage.coverage import evaluate
	8
	9	FIXTURES = Path(__file__).parent / "fixtures"
	10	GITHUB = FIXTURES / "github_audit_acme_2026-01-01.json"
	11	AWS = FIXTURES / "aws_audit_acme_2026-01-01.json"
	12
	13
	14	def _crosswalk(frameworks):
	15	obs = corpus.load_corpus([GITHUB, AWS])
	16	cats = catalog.load_frameworks(frameworks)
	17	return crosswalk.build(evaluate(cats, obs))
	18
	19
	20	def test_item_spans_multiple_frameworks():
	21	xw = _crosswalk(["SOC2", "ISO", "NIST"])
	22	twofa = next(i for i in xw.items if i.rule_id == "github.org.require-2fa")
	23	# 2FA maps to SOC2:CC6.1, ISO:A.5.17, NIST:IA-2.
	24	assert set(twofa.frameworks) == {"SOC2", "ISO", "NIST"}
	25	assert "SOC2:CC6.1" in twofa.controls
	26
	27
	28	def test_items_sorted_by_leverage():
	29	xw = _crosswalk(["SOC2", "ISO", "NIST"])
	30	counts = [i.count for i in xw.items]
	31	assert counts == sorted(counts, reverse=True)
	32
	33
	34	def test_minimal_cover_reaches_full_universe():
	35	xw = _crosswalk(["SOC2", "ISO", "NIST"])
	36	assert xw.cover # non-empty
	37	assert xw.cover[-1].cumulative == xw.universe_size
	38	assert xw.cover[-1].cumulative_pct == 100.0
	39
	40
	41	def test_cover_is_monotonic_and_no_wasted_picks():
	42	xw = _crosswalk(["SOC2"])
	43	cumulative = [s.cumulative for s in xw.cover]
	44	assert cumulative == sorted(cumulative)
	45	assert all(s.new_controls > 0 for s in xw.cover) # greedy never picks a no-op
	46
	47
	48	def test_markdown_has_both_sections():
	49	md = crosswalk.render_markdown(_crosswalk(["SOC2", "ISO"]))
	50	assert "## Minimal evidence set" in md
	51	assert "## Evidence leverage" in md
	52	assert "github.org.require-2fa" in md
	53
	54
	55	def test_json_structure():
	56	doc = json.loads(crosswalk.render_json(_crosswalk(["SOC2", "ISO"])))
	57	assert doc["universe_size"] > 0
	58	assert "minimal_evidence_set" in doc
	59	assert all("controls" in e for e in doc["evidence"])
	60
	61
	62	def test_html_is_self_contained():
	63	html = crosswalk.render_html(_crosswalk(["SOC2", "ISO", "NIST"]))
	64	assert html.startswith("<!doctype html>")
	65	assert "<style>" in html
	66	assert "http://" not in html and "https://" not in html
	67	assert "github.org.require-2fa" in html

tests/test_reporters.py added +55

@@ -0,0 +1,55 @@
	1	"""Tests for the Markdown, HTML, JSON, and SoA renderers."""
	2
	3	import json as _json
	4	from pathlib import Path
	5
	6	from control_coverage import catalog, corpus, reporters, scope
	7	from control_coverage.coverage import evaluate
	8
	9	FIXTURES = Path(__file__).parent / "fixtures"
	10
	11
	12	def _report():
	13	obs = corpus.load_corpus([FIXTURES / "github_audit_acme_2026-01-01.json"])
	14	cats = catalog.load_frameworks(["SOC2"])
	15	scp = scope.load(FIXTURES / "scope.yaml")
	16	return evaluate(cats, obs, scope=scp, subject="Acme", generated_at="2026-01-01")
	17
	18
	19	def test_markdown_has_summary_and_blind_spots():
	20	md = reporters.render(_report(), "md")
	21	assert "# Control Coverage — Acme" in md
	22	assert "## Summary" in md
	23	assert "## Blind spots" in md
	24	assert "CC1.1" in md # a blind spot is listed
	25
	26
	27	def test_json_is_valid_and_structured():
	28	doc = _json.loads(reporters.render(_report(), "json"))
	29	assert doc["subject"] == "Acme"
	30	soc2 = doc["frameworks"][0]
	31	assert soc2["framework"] == "SOC2"
	32	assert soc2["coverage_pct"] >= 0
	33	states = {c["state"] for c in soc2["controls"]}
	34	assert "unaddressed" in states
	35
	36
	37	def test_html_is_self_contained():
	38	html = reporters.render(_report(), "html")
	39	assert html.startswith("<!doctype html>")
	40	assert "<style>" in html
	41	assert "http://" not in html and "https://" not in html # no external assets
	42
	43
	44	def test_soa_lists_applicability_and_status():
	45	soa = reporters.render(_report(), "soa")
	46	assert "Statement of Applicability" in soa
	47	assert "Applicable" in soa
	48	assert "Implemented" in soa or "Not evidenced" in soa
	49
	50
	51	def test_unknown_format_raises():
	52	import pytest
	53
	54	with pytest.raises(ValueError, match="unknown format"):
	55	reporters.render(_report(), "pdf")

tests/test_scope.py added +65

@@ -0,0 +1,65 @@
	1	"""Tests for scope / Statement of Applicability parsing."""
	2
	3	from pathlib import Path
	4
	5	import pytest
	6
	7	from control_coverage import scope
	8
	9	FIXTURES = Path(__file__).parent / "fixtures"
	10
	11
	12	def test_loads_exclusions_and_owners():
	13	scp = scope.load(FIXTURES / "scope.yaml")
	14	assert scp.subject == "Acme Production"
	15	assert scp.frameworks == ["SOC2", "ISO"]
	16	assert scp.excluded("ISO:A.7.1")
	17	assert "cloud-hosted" in scp.reason("ISO:A.7.1")
	18	assert scp.owner("SOC2:CC6.1") == "platform-team"
	19
	20
	21	def test_exclusion_without_reason_is_rejected(tmp_path):
	22	bad = tmp_path / "bad.yaml"
	23	bad.write_text("exclusions:\n - control: ISO:A.5.7\n")
	24	with pytest.raises(ValueError, match="needs a non-empty 'reason'"):
	25	scope.load(bad)
	26
	27
	28	def test_exclusion_missing_control_key_is_rejected(tmp_path):
	29	bad = tmp_path / "bad.yaml"
	30	bad.write_text("exclusions:\n - reason: no control key\n")
	31	with pytest.raises(ValueError, match="must be a mapping with a 'control' key"):
	32	scope.load(bad)
	33
	34
	35	def test_family_exclusion_parses(tmp_path):
	36	f = tmp_path / "s.yaml"
	37	f.write_text(
	38	"exclude_families:\n"
	39	" - {framework: SOC2, family: Privacy, reason: 'Not in the audit scope.'}\n"
	40	)
	41	scp = scope.load(f)
	42	assert scp.family_excluded("SOC2", "Privacy")
	43	assert "audit scope" in scp.family_reason("SOC2", "Privacy")
	44	assert not scp.family_excluded("SOC2", "Availability")
	45
	46
	47	def test_family_exclusion_needs_reason(tmp_path):
	48	f = tmp_path / "s.yaml"
	49	f.write_text("exclude_families:\n - {framework: SOC2, family: Privacy}\n")
	50	with pytest.raises(ValueError, match="needs a non-empty 'reason'"):
	51	scope.load(f)
	52
	53
	54	def test_family_exclusion_needs_framework_and_family(tmp_path):
	55	f = tmp_path / "s.yaml"
	56	f.write_text("exclude_families:\n - {family: Privacy, reason: x}\n")
	57	with pytest.raises(ValueError, match="'framework' and 'family'"):
	58	scope.load(f)
	59
	60
	61	def test_empty_scope_excludes_nothing():
	62	scp = scope.empty()
	63	assert not scp.excluded("ISO:A.7.1")
	64	assert not scp.family_excluded("SOC2", "Privacy")
	65	assert scp.frameworks == []

tests/test_trend.py added +75

@@ -0,0 +1,75 @@
	1	"""Tests for coverage trend (diffing two corpora)."""
	2
	3	from pathlib import Path
	4
	5	from control_coverage import catalog, corpus, trend
	6	from control_coverage.coverage import evaluate
	7
	8	FIXTURES = Path(__file__).parent / "fixtures"
	9	GITHUB = FIXTURES / "github_audit_acme_2026-01-01.json"
	10	AWS = FIXTURES / "aws_audit_acme_2026-01-01.json"
	11	BASELINE = FIXTURES / "baseline_github.json"
	12
	13
	14	def _cov(paths):
	15	obs = corpus.load_corpus(paths)
	16	cats = catalog.load_frameworks(["SOC2"])
	17	return evaluate(cats, obs)
	18
	19
	20	def _compare():
	21	return trend.compare(_cov([BASELINE]), _cov([GITHUB, AWS]))
	22
	23
	24	def _delta(fc, control_id):
	25	return next(d for d in fc.deltas if d.id == control_id)
	26
	27
	28	def test_categories_reflect_state_movement():
	29	fc = _compare().frameworks[0]
	30	assert _delta(fc, "CC6.1").category == trend.IMPROVED # failing -> supported
	31	assert _delta(fc, "CC6.3").category == trend.UNCHANGED # failing -> failing
	32	assert _delta(fc, "CC6.6").category == trend.GAINED # unaddressed -> failing
	33	assert _delta(fc, "CC7.2").category == trend.GAINED # unaddressed -> supported
	34	assert _delta(fc, "CC9.2").category == trend.LOST # supported -> unaddressed
	35
	36
	37	def test_counts_and_coverage_delta():
	38	fc = _compare().frameworks[0]
	39	c = fc.counts
	40	assert c[trend.IMPROVED] == 1
	41	assert c[trend.GAINED] == 3 # CC6.6, CC7.2, CC8.1
	42	assert c[trend.LOST] == 1
	43	assert c[trend.REGRESSED] == 0
	44	assert fc.coverage_delta == round(fc.new_coverage_pct - fc.old_coverage_pct, 1)
	45	assert fc.new_coverage_pct > fc.old_coverage_pct
	46
	47
	48	def test_regressions_count_lost_and_regressed():
	49	report = _compare()
	50	assert report.total_regressions == 1 # the single LOST control
	51
	52
	53	def test_markdown_lists_changes_only():
	54	md = trend.render_markdown(_compare())
	55	assert "# Coverage Trend" in md
	56	assert "CC9.2" in md # a lost control appears
	57	assert "CC1.1" not in md # an unchanged blind spot does not
	58
	59
	60	def test_json_omits_unchanged():
	61	import json
	62
	63	doc = json.loads(trend.render_json(_compare()))
	64	changes = doc["frameworks"][0]["changes"]
	65	ids = {c["id"] for c in changes}
	66	assert "CC9.2" in ids
	67	assert "CC1.1" not in ids
	68
	69
	70	def test_html_is_self_contained():
	71	html = trend.render_html(_compare())
	72	assert html.startswith("<!doctype html>")
	73	assert "<style>" in html
	74	assert "http://" not in html and "https://" not in html
	75	assert "CC9.2" in html # a changed control shows up

Commit 4187ceff35

Commit `4187ceff35`