docs/guide/09-auditing.org

  1#+TITLE: Auditing a corpus
  2#+DESCRIPTION: Find out what a tool will make of your writing before you trust it with it.
  3#+LEDE: Construct frequencies, unknown-name census, and no document text in the output.
  4
  5#+BEGIN_SRC sh
  6orgo audit ~/notes
  7#+END_SRC
  8
  9The audit answers two questions about a body of org files:
 10
 111. *Coverage.* Of the constructs this corpus uses, which are supported? A construct that
 12   is common here and unsupported is a problem with the tool's scope, not with your
 13   writing.
 142. *Blind spots.* Which names appear that orgo has no opinion about at all? These are
 15   the dangerous ones — not "known unsupported" but unknown.
 16
 17* Reading the output
 18
 19#+BEGIN_EXAMPLE
 20corpus: 180 file(s), 29742 line(s)
 21
 22CONSTRUCTS (by frequency)
 23     construct                            uses   files  first seen
 24IN   list item                            1309     111  blog/2018-11-28-aes-encryption.org:53
 25IN   heading                              1148     176  blog/2018-11-28-aes-encryption.org:7
 26IN   verbatim                             1048     130  blog/2018-11-28-aes-encryption.org:79
 27...
 28IN   table formula (#+TBLFM:)                4       1  blog/2024-08-11-org-mode-features.org:191
 29IN   special block                           1       1  blog/2026-03-03-auditing-aws-s3.org:50
 30
 31coverage: 9002 in-scope use(s) (100.0%), 0 out-of-scope (0.0%)
 32
 33KEYWORDS
 34    SLUG                                 180     180  blog/2018-11-28-aes-encryption.org:4
 35    TITLE                                180     180  blog/2018-11-28-aes-encryption.org:2
 36—   LEDE                                  14      14  blog/2018-11-28-aes-encryption.org:3
 37...
 38#+END_EXAMPLE
 39
 40- =IN= is supported; =OUT= is excluded by design and degrades as described in
 41  [[file:05-org-support.org][Org support]].
 42- The *coverage* line is the number to look at first.
 43- =???= marks a name orgo does not recognise at all. That is the blind-spot signal —
 44  not "known unsupported", but unknown — and this corpus has none. Block names never
 45  carry it: an unrecognised one is still a special block, and still renders. Keyword
 46  names never carry it either — see below.
 47- =—= marks a keyword with no dedicated handling. It is not a gap: the keyword reaches
 48  your layout as ={{ page.keywords.<name> }}=, which is the designed behaviour, so the
 49  marker tells you which of your keywords orgo reads by name and which rely on that
 50  pass-through.
 51
 52Four censuses follow the construct table: every distinct =#+KEYWORD:=, block type,
 53drawer name and link scheme in the corpus. A =???= in any of them is worth a look.
 54
 55* It never prints your writing
 56
 57Names, counts and =file:line= locations only. That is a deliberate constraint so that an
 58audit of private notes — work notes, a journal — is safe to paste into an issue or share
 59with someone helping you.
 60
 61* Why it is a separate scanner
 62
 63The audit deliberately does *not* reuse the parser. Auditing with the parser could only
 64ever find constructs the parser already knows about, which is exactly the wrong
 65instrument for the second question: it would report a blind spot as clean.
 66
 67* Comparing against Emacs
 68
 69The second half of the same idea is a differential test suite. =cargo test --test oracle=
 70exports each fixture with org's own HTML exporter through =emacs --batch=, reduces both
 71outputs to a semantic skeleton, and *snapshots the disagreement*.
 72
 73#+BEGIN_SRC sh
 74cargo test --test oracle
 75#+END_SRC
 76
 77Snapshotting rather than asserting agreement is deliberate: a checked-in divergence
 78report gets reviewed and shows up in code review, where a permanently red test gets
 79ignored. Three invariants /are/ asserted outright — heading structure, list nesting and
 80source-block text — and all three hold.
 81
 82The suite skips cleanly with no Emacs installed, so a machine without it still gets a
 83green run; it simply measures one thing less.
 84
 85* Using the audit before a migration
 86
 87#+BEGIN_SRC sh
 88# What is in there?
 89orgo audit ~/notes
 90
 91# Build it and see what the builder itself complains about.
 92orgo build ~/notes -o /tmp/preview --strict
 93
 94# Look at the result.
 95orgo serve ~/notes -o /tmp/preview
 96#+END_SRC
 97
 98=--strict= surfaces broken internal links and malformed constructs as failures rather
 99than warnings, which is the fastest way to find the handful of files that need attention
100before you commit to anything.