krz/orgo

 1#+TITLE: Auditing a corpus
 2#+DESCRIPTION: Find out what a tool will make of your writing before you trust it with it.
 3#+LEDE: Construct frequencies, unknown-name census, and no document text in the output.
 4
 5#+BEGIN_SRC sh
 6orgo audit ~/notes
 7#+END_SRC
 8
 9The audit answers two questions about a body of org files:
10
111. *Coverage.* Of the constructs this corpus uses, which are supported? A construct that
12   is common here and unsupported is a problem with the tool's scope, not with your
13   writing.
142. *Blind spots.* Which names appear that orgo has no opinion about at all? These are
15   the dangerous ones — not "known unsupported" but unknown.
16
17* Reading the output
18
19#+BEGIN_EXAMPLE
20corpus: 180 file(s), 29742 line(s)
21
22CONSTRUCTS (by frequency)
23     construct                            uses   files  first seen
24IN   list item                            1309     111  blog/2018-11-28-aes-encryption.org:53
25IN   heading                              1148     176  blog/2018-11-28-aes-encryption.org:7
26IN   verbatim                             1048     130  blog/2018-11-28-aes-encryption.org:79
27...
28IN   table formula (#+TBLFM:)                4       1  blog/2024-08-11-org-mode-features.org:191
29IN   special block                           1       1  blog/2026-03-03-auditing-aws-s3.org:50
30
31coverage: 9002 in-scope use(s) (100.0%), 0 out-of-scope (0.0%)
32
33KEYWORDS
34   SLUG                                  180     180  blog/2018-11-28-aes-encryption.org:4
35   TITLE                                 180     180  blog/2018-11-28-aes-encryption.org:2
36...
37#+END_EXAMPLE
38
39- =IN= is supported; =OUT= is excluded by design and degrades as described in
40  [[file:05-org-support.org][Org support]].
41- The *coverage* line is the number to look at first.
42- =???= marks a name orgo does not recognise at all. That is the blind-spot signal —
43  not "known unsupported", but unknown — and this corpus has none. Block names never
44  carry it: an unrecognised one is still a special block, and still renders.
45
46Four censuses follow the construct table: every distinct =#+KEYWORD:=, block type,
47drawer name and link scheme in the corpus. A =???= in any of them is worth a look.
48
49* It never prints your writing
50
51Names, counts and =file:line= locations only. That is a deliberate constraint so that an
52audit of private notes — work notes, a journal — is safe to paste into an issue or share
53with someone helping you.
54
55* Why it is a separate scanner
56
57The audit deliberately does *not* reuse the parser. Auditing with the parser could only
58ever find constructs the parser already knows about, which is exactly the wrong
59instrument for the second question: it would report a blind spot as clean.
60
61* Comparing against Emacs
62
63The second half of the same idea is a differential test suite. =cargo test --test oracle=
64exports each fixture with org's own HTML exporter through =emacs --batch=, reduces both
65outputs to a semantic skeleton, and *snapshots the disagreement*.
66
67#+BEGIN_SRC sh
68cargo test --test oracle
69#+END_SRC
70
71Snapshotting rather than asserting agreement is deliberate: a checked-in divergence
72report gets reviewed and shows up in code review, where a permanently red test gets
73ignored. Three invariants /are/ asserted outright — heading structure, list nesting and
74source-block text — and all three hold.
75
76The suite skips cleanly with no Emacs installed, so a machine without it still gets a
77green run; it simply measures one thing less.
78
79* Using the audit before a migration
80
81#+BEGIN_SRC sh
82# What is in there?
83orgo audit ~/notes
84
85# Build it and see what the builder itself complains about.
86orgo build ~/notes -o /tmp/preview --strict
87
88# Look at the result.
89orgo serve ~/notes -o /tmp/preview
90#+END_SRC
91
92=--strict= surfaces broken internal links and malformed constructs as failures rather
93than warnings, which is the fastest way to find the handful of files that need attention
94before you commit to anything.