krz/orgo

 1#+TITLE: Auditing a corpus
 2#+DESCRIPTION: Find out what a tool will make of your writing before you trust it with it.
 3#+LEDE: Construct frequencies, unknown-name census, and no document text in the output.
 4
 5#+BEGIN_SRC sh
 6orgo audit ~/notes
 7#+END_SRC
 8
 9The audit answers two questions about a body of org files:
10
111. *Coverage.* Of the constructs this corpus uses, which are supported? A construct that
12   is common here and unsupported is a problem with the tool's scope, not with your
13   writing.
142. *Blind spots.* Which names appear that orgo has no opinion about at all? These are
15   the dangerous ones — not "known unsupported" but unknown.
16
17* Reading the output
18
19#+BEGIN_EXAMPLE
20corpus: 179 file(s), 29258 line(s)
21
22CONSTRUCTS (by frequency)
23     construct                            uses   files  first seen
24IN   list item                            1282     109  blog/2018-11-28-aes-encryption.org:53
25IN   heading                              1128     174  blog/2018-11-28-aes-encryption.org:7
26IN   source block                          932     121  blog/2018-11-28-cpp-compiler.org:17
27...
28OUT  table formula (#+TBLFM:)                4       1  blog/2024-08-11-org-mode-features.org:191
29OUT  entity (\name)                          3       3  blog/2024-04-06-convert-onenote.org:37
30
31coverage: 8854 in-scope use(s) (99.9%), 8 out-of-scope (0.1%)
32
33KEYWORDS
34   TITLE                                 179     179  blog/2018-11-28-aes-encryption.org:2
35??? SLUG                                  178     178  blog/2018-11-28-aes-encryption.org:4
36   DESCRIPTION                           176     176  blog/2018-11-28-aes-encryption.org:3
37...
38#+END_EXAMPLE
39
40- =IN= is supported; =OUT= is excluded by design and degrades as described in
41  [[file:05-org-support.org][Org support]].
42- The *coverage* line is the number to look at first.
43- =???= marks a name orgo does not recognise at all — in this example =#+SLUG:=, from
44  a run made before it was supported.
45
46Four censuses follow the construct table: every distinct =#+KEYWORD:=, block type,
47drawer name and link scheme in the corpus. A =???= in any of them is worth a look.
48
49* It never prints your writing
50
51Names, counts and =file:line= locations only. That is a deliberate constraint so that an
52audit of private notes — work notes, a journal — is safe to paste into an issue or share
53with someone helping you.
54
55* Why it is a separate scanner
56
57The audit deliberately does *not* reuse the parser. Auditing with the parser could only
58ever find constructs the parser already knows about, which is exactly the wrong
59instrument for the second question: it would report a blind spot as clean.
60
61* Comparing against Emacs
62
63The second half of the same idea is a differential test suite. =cargo test --test oracle=
64exports each fixture with org's own HTML exporter through =emacs --batch=, reduces both
65outputs to a semantic skeleton, and *snapshots the disagreement*.
66
67#+BEGIN_SRC sh
68cargo test --test oracle
69#+END_SRC
70
71Snapshotting rather than asserting agreement is deliberate: a checked-in divergence
72report gets reviewed and shows up in code review, where a permanently red test gets
73ignored. Three invariants /are/ asserted outright — heading structure, list nesting and
74source-block text — and all three hold.
75
76The suite skips cleanly with no Emacs installed, so a machine without it still gets a
77green run; it simply measures one thing less.
78
79* Using the audit before a migration
80
81#+BEGIN_SRC sh
82# What is in there?
83orgo audit ~/notes
84
85# Build it and see what the builder itself complains about.
86orgo build ~/notes -o /tmp/preview --strict
87
88# Look at the result.
89orgo serve ~/notes -o /tmp/preview
90#+END_SRC
91
92=--strict= surfaces broken internal links and malformed constructs as failures rather
93than warnings, which is the fastest way to find the handful of files that need attention
94before you commit to anything.