krz/orgo

Lightning fast org-mode static site generator. fast go org-mode static-site-generator

Commit bb6ef333ca

bb6ef333ca9e4b42280a2b860e722e76ff9c222b

parent: 94205548ab

Unsigned ci/pages: success ci/test: success

cmc <hello@cleberg.net> · 2026-10-18 20:22 UTC 

Track org 9.8 in the Emacs oracle (!11)

`cargo test --locked --test oracle` failed four snapshots on Emacs 31.1 /
org 9.8.7, identically on `main`. Neither cause is in orgo.

**org 9.8 converged on `<pre><code>`.** `ox-html.el` now wraps source
blocks the way orgo already did, so the `pre-code` divergence stopped
firing: `core` 87.9% → 91.4%, `blocks` 87.0% → 93.6%, `elements` 78.0% →
80.5%. The rule stays, because org 9.7 and earlier write a bare `<pre>`.

**`ox-html` 9.8.7 emits a malformed entity.** The heading tag prefix is
written `&nbsp;&nbsp;&nbsp` — the last reference unterminated
(`ox-html.el:2915`). `decode_entities` required the semicolon and left it
literal, so a heading with tags reduced to `A Subheading &nbsptag1 tag2`
against orgo's `A Subheading tag1 tag2`. HTML5 decodes the legacy named
set without the semicolon and a browser renders three non-breaking
spaces; the skeleton now does the same. `minimal` is back to its
committed 38/42 (90.5%).

**The `semantic-time` rule was hiding both.** It accepted any hunk where
`ours` was tags-or-text and `theirs` was timestamp-or-text — and any text
line satisfies both, so every prose difference matched. The `&nbsp`
mismatch therefore arrived labelled as a deliberate rendering choice and
`every_divergence_from_emacs_is_deliberate` kept passing. Guarded with
`any` on both sides.

That guard exposed a second divergence the rule had also been absorbing:
orgo keeps heading priority cookies, org drops them
(`org-export-with-priority` defaults to nil). It is the policy the module
docs already claim — "heading decoration is a policy difference" — but no
rule recorded it. Added as `heading-priority-cookie`.

**Reports now name the Emacs and org version they ran against.** The
oracle is whatever Emacs is on the machine, so a snapshot without that is
not interpretable, and a version bump silently rewrites the agreement
percentage — which is how this sat on `main`. The cost is that the
snapshots churn on an Emacs upgrade; the diff then says `oracle: emacs
31.1, org 9.8.7` changed, which is the thing you need to know. Pinning
Emacs instead would make it a build dependency, which the suite
deliberately avoids (it skips when Emacs is absent, as CI does).

`cargo test --locked` green, 15/15 oracle tests, `unexplained: 0` on all
twelve fixtures. `cargo clippy --all-targets --locked -- -D warnings`
clean. New unit test `unterminated_named_references_decode_like_a_browser`
pins the entity behaviour without needing any particular Emacs.

Layout: unified · split

src/skeleton.rs +47 −27
@@ -31,6 +31,19 @@ const IGNORED: &[&str] = &["div", "span"];
31/// cosmetic (`org-ul`); `href` and `src` are the content. 31/// cosmetic (`org-ul`); `href` and `src` are the content.
32const KEPT_ATTRS: &[&str] = &["href", "src"]; 32const KEPT_ATTRS: &[&str] = &["href", "src"];
33 33
34/// The named references this reduction understands. HTML5 decodes a legacy set of them
35/// without the closing semicolon, and org relies on that: `ox-html` 9.8 writes the
36/// heading tag prefix as `&nbsp;&nbsp;&nbsp`. No name here is a prefix of another, so the
37/// longest match is the only match.
38const NAMED: &[(&str, char)] = &[
39 ("amp", '&'),
40 ("lt", '<'),
41 ("gt", '>'),
42 ("quot", '"'),
43 ("apos", '\''),
44 ("nbsp", '\u{a0}'),
45];
46
34/// HTML void elements, which never emit a close event. 47/// HTML void elements, which never emit a close event.
35const VOID: &[&str] = &[ 48const VOID: &[&str] = &[
36 "br", "hr", "img", "input", "meta", "link", "col", "area", "base", "source", "wbr", 49 "br", "hr", "img", "input", "meta", "link", "col", "area", "base", "source", "wbr",
@@ -151,39 +164,46 @@ pub fn decode_entities(s: &str) -> String {
151 while let Some(amp) = rest.find('&') { 164 while let Some(amp) = rest.find('&') {
152 out.push_str(&rest[..amp]); 165 out.push_str(&rest[..amp]);
153 let tail = &rest[amp..]; 166 let tail = &rest[amp..];
154 let Some(semi) = tail.find(';').filter(|s| *s <= 12) else { 167 match entity_at(tail) {
155 out.push('&'); 168 Some((c, len)) => {
156 rest = &tail[1..]; 169 // A non-breaking space is a space for comparison purposes.
157 continue; 170 out.push(if c == '\u{a0}' { ' ' } else { c });
158 }; 171 rest = &tail[len..];
159 let entity = &tail[1..semi]; 172 }
160 let decoded = match entity {
161 "amp" => Some('&'),
162 "lt" => Some('<'),
163 "gt" => Some('>'),
164 "quot" => Some('"'),
165 "apos" => Some('\''),
166 "nbsp" => Some(' '),
167 _ => entity
168 .strip_prefix('#')
169 .and_then(|n| match n.strip_prefix(['x', 'X']) {
170 Some(hex) => u32::from_str_radix(hex, 16).ok(),
171 None => n.parse::<u32>().ok(),
172 })
173 .and_then(char::from_u32),
174 };
175 match decoded {
176 // A non-breaking space is a space for comparison purposes.
177 Some('\u{a0}') => out.push(' '),
178 Some(c) => out.push(c),
179 None => { 173 None => {
180 out.push('&'); 174 out.push('&');
181 rest = &tail[1..]; 175 rest = &tail[1..];
182 continue;
183 } 176 }
184 } 177 }
185 rest = &tail[semi + 1..];
186 } 178 }
187 out.push_str(rest); 179 out.push_str(rest);
188 out 180 out
189} 181}
182
183/// The reference at the start of `tail`, which begins with `&`: the character it denotes
184/// and the bytes it occupies. The semicolon is optional for [`NAMED`], so `&nbsptag1`
185/// reads as a non-breaking space followed by `tag1`, the same way a browser reads it.
186fn entity_at(tail: &str) -> Option<(char, usize)> {
187 if let Some(semi) = tail.find(';').filter(|s| *s <= 12) {
188 if let Some(c) = named(&tail[1..semi]).or_else(|| numeric(&tail[1..semi])) {
189 return Some((c, semi + 1));
190 }
191 }
192 NAMED
193 .iter()
194 .find(|(name, _)| tail[1..].starts_with(name))
195 .map(|(name, c)| (*c, 1 + name.len()))
196}
197
198fn named(name: &str) -> Option<char> {
199 NAMED.iter().find(|(n, _)| *n == name).map(|(_, c)| *c)
200}
201
202fn numeric(name: &str) -> Option<char> {
203 name.strip_prefix('#')
204 .and_then(|n| match n.strip_prefix(['x', 'X']) {
205 Some(hex) => u32::from_str_radix(hex, 16).ok(),
206 None => n.parse::<u32>().ok(),
207 })
208 .and_then(char::from_u32)
209}
tests/oracle.rs +80 −4
@@ -45,6 +45,27 @@ fn emacs_available() -> bool {
45 .unwrap_or(false) 45 .unwrap_or(false)
46} 46}
47 47
48/// Emacs and org, as the `+` side of every report below. The ground truth moves: org
49/// 9.8 began wrapping source blocks in `<pre><code>`, which orgo already did, so three
50/// of these snapshots gained several points without orgo changing. Recorded so that
51/// shows in the diff as what it is rather than reading as a regression.
52fn oracle_version() -> &'static str {
53 static VERSION: std::sync::OnceLock<String> = std::sync::OnceLock::new();
54 VERSION.get_or_init(|| {
55 let output = Command::new("emacs")
56 .args([
57 "-Q",
58 "--batch",
59 "--eval",
60 "(progn (require 'org) \
61 (princ (format \"emacs %s, org %s\" emacs-version (org-version))))",
62 ])
63 .output()
64 .expect("run emacs");
65 String::from_utf8_lossy(&output.stdout).trim().to_string()
66 })
67}
68
48/// Export a fixture with org's own HTML exporter. 69/// Export a fixture with org's own HTML exporter.
49fn org_export(fixture: &str) -> String { 70fn org_export(fixture: &str) -> String {
50 let root = manifest_dir(); 71 let root = manifest_dir();
@@ -105,6 +126,22 @@ fn text_of(line: &str) -> Option<&str> {
105 line.strip_prefix('"')?.strip_suffix('"') 126 line.strip_prefix('"')?.strip_suffix('"')
106} 127}
107 128
129/// The one text run a hunk consists of, if that is all it is.
130fn one_text(lines: &[String]) -> Option<&str> {
131 match lines {
132 [line] => text_of(line),
133 _ => None,
134 }
135}
136
137/// `TODO [#A] Write the parser` → `TODO Write the parser`.
138fn without_priority(text: &str) -> Option<String> {
139 let open = text.find("[#")?;
140 let close = open + text[open..].find(']')?;
141 let joined = format!("{}{}", &text[..open], text[close + 1..].trim_start());
142 Some(joined.trim().to_string())
143}
144
108/// Does this text run contain an org timestamp, `<2024-01-15 Mon>` or `[2024-01-15]`? 145/// Does this text run contain an org timestamp, `<2024-01-15 Mon>` or `[2024-01-15]`?
109fn has_timestamp(line: &str) -> bool { 146fn has_timestamp(line: &str) -> bool {
110 let t = match text_of(line) { 147 let t = match text_of(line) {
@@ -120,14 +157,31 @@ const DELIBERATE: &[Deliberate] = &[
120 Deliberate { 157 Deliberate {
121 name: "semantic-time", 158 name: "semantic-time",
122 matches: |ours, theirs| { 159 matches: |ours, theirs| {
123 !ours.is_empty() 160 // Both sides are allowed to carry text, so without the `any` guards this
161 // rule matches *any* hunk of differing prose and files it under a rendering
162 // choice nobody made. That is how org 9.8's malformed `&nbsp` in the heading
163 // tag prefix arrived here labelled `semantic-time`.
164 ours.iter().any(|l| is_tag(l, &["time"]))
124 && ours 165 && ours
125 .iter() 166 .iter()
126 .all(|l| is_tag(l, &["time"]) || text_of(l).is_some()) 167 .all(|l| is_tag(l, &["time"]) || text_of(l).is_some())
168 && theirs.iter().any(|l| has_timestamp(l))
127 && theirs.iter().all(|l| has_timestamp(l) || text_of(l).is_some()) 169 && theirs.iter().all(|l| has_timestamp(l) || text_of(l).is_some())
128 }, 170 },
129 in_notes_only: false, 171 in_notes_only: false,
130 }, 172 },
173 // The priority cookie stays in the heading: the author wrote it, and a reader
174 // skimming a task list wants to see it. Org drops it, because
175 // `org-export-with-priority` defaults to nil. Heading decoration is a policy
176 // difference, not a structural one.
177 Deliberate {
178 name: "heading-priority-cookie",
179 matches: |ours, theirs| match (one_text(ours), one_text(theirs)) {
180 (Some(ours), Some(theirs)) => without_priority(ours).as_deref() == Some(theirs),
181 _ => false,
182 },
183 in_notes_only: false,
184 },
131 // `<figure>`/`<figcaption>` instead of two paragraphs in a div. 185 // `<figure>`/`<figcaption>` instead of two paragraphs in a div.
132 Deliberate { 186 Deliberate {
133 name: "figure-element", 187 name: "figure-element",
@@ -150,8 +204,10 @@ const DELIBERATE: &[Deliberate] = &[
150 }, 204 },
151 in_notes_only: false, 205 in_notes_only: false,
152 }, 206 },
153 // `<pre><code>` instead of a bare `<pre>`: the nested element is what every syntax 207 // `<pre><code>`: the nested element is what every syntax highlighter and every
154 // highlighter and every reader's stylesheet expects. 208 // reader's stylesheet expects. Org 9.8 emits it too, so this no longer fires — kept
209 // because org 9.7 and earlier write a bare `<pre>`, and the oracle is whatever Emacs
210 // is on the machine.
155 Deliberate { 211 Deliberate {
156 name: "pre-code", 212 name: "pre-code",
157 matches: |ours, theirs| { 213 matches: |ours, theirs| {
@@ -407,7 +463,11 @@ fn compare(fixture: &str) -> Option<String> {
407 } 463 }
408 let ours = skeleton(&our_export(fixture)); 464 let ours = skeleton(&our_export(fixture));
409 let theirs = skeleton(&org_export(fixture)); 465 let theirs = skeleton(&org_export(fixture));
410 Some(divergence(&ours, &theirs)) 466 Some(format!(
467 "oracle: {}\n{}",
468 oracle_version(),
469 divergence(&ours, &theirs)
470 ))
411} 471}
412 472
413macro_rules! oracle_test { 473macro_rules! oracle_test {
@@ -480,6 +540,22 @@ fn every_divergence_from_emacs_is_deliberate() {
480// Invariants that must hold against the oracle, not merely be snapshotted 540// Invariants that must hold against the oracle, not merely be snapshotted
481// --------------------------------------------------------------------------- 541// ---------------------------------------------------------------------------
482 542
543/// org 9.8 writes the heading tag prefix as `&nbsp;&nbsp;&nbsp` — the last reference
544/// unterminated. HTML5 decodes the legacy named set without the semicolon, so a browser
545/// reads that as three non-breaking spaces; the skeleton has to as well, or a heading
546/// carrying tags reads as `A Subheading &nbsptag1 tag2` and the whole report shifts.
547/// Asserted here rather than left to the snapshots because it must hold whichever Emacs
548/// is installed, including none.
549#[test]
550fn unterminated_named_references_decode_like_a_browser() {
551 assert_eq!(
552 decode_entities("A Subheading&nbsp;&nbsp;&nbsptag1&nbsp;tag2"),
553 "A Subheading tag1 tag2"
554 );
555 assert_eq!(decode_entities("a &amp b"), "a & b");
556 assert_eq!(decode_entities("Q&A"), "Q&A");
557}
558
483/// How many headings a document has and at what depth is the shape of the document. 559/// How many headings a document has and at what depth is the shape of the document.
484/// Getting it wrong reorganizes someone's writing, so it is asserted rather than 560/// Getting it wrong reorganizes someone's writing, so it is asserted rather than
485/// snapshotted. Heading *decoration* (priority cookies, tag markup) is a policy 561/// snapshotted. Heading *decoration* (priority cookies, tag markup) is a policy
tests/snapshots/oracle__oracle_blocks.snap +9 −8
@@ -2,8 +2,9 @@
2source: tests/oracle.rs 2source: tests/oracle.rs
3expression: report 3expression: report
4--- 4---
5agreement: 67/77 skeleton lines (87.0%) 5oracle: emacs 31.1, org 9.8.7
6deliberate: 11 line(s) — pre-code ×6, semantic-emphasis ×4, verse-trailing-break ×1 6agreement: 73/78 skeleton lines (93.6%)
7deliberate: 5 line(s) — semantic-emphasis ×4, verse-trailing-break ×1
7unexplained: 0 line(s) 8unexplained: 0 line(s)
8(- orgo, + emacs, ~ a difference we mean to have) 9(- orgo, + emacs, ~ a difference we mean to have)
9 10
@@ -44,14 +45,14 @@ unexplained: 0 line(s)
44 "Source" 45 "Source"
45 </h2> 46 </h2>
46 <pre> 47 <pre>
47~ pre-code (1 line(s)) 48 <code>
48 "def greet(name): return f\"hello {name}\"" 49 "def greet(name): return f\"hello {name}\""
49~ pre-code (1 line(s)) 50 </code>
50 </pre> 51 </pre>
51 <pre> 52 <pre>
52~ pre-code (1 line(s)) 53 <code>
53 "plain block, no language" 54 "plain block, no language"
54~ pre-code (1 line(s)) 55 </code>
55 </pre> 56 </pre>
56 <h2> 57 <h2>
57 "Nested" 58 "Nested"
@@ -61,9 +62,9 @@ unexplained: 0 line(s)
61 "A quote containing a source block:" 62 "A quote containing a source block:"
62 </p> 63 </p>
63 <pre> 64 <pre>
64~ pre-code (1 line(s)) 65 <code>
65 "echo hi" 66 "echo hi"
66~ pre-code (1 line(s)) 67 </code>
67 </pre> 68 </pre>
68 </blockquote> 69 </blockquote>
69 <h2> 70 <h2>
tests/snapshots/oracle__oracle_core.snap +5 −4
@@ -2,8 +2,9 @@
2source: tests/oracle.rs 2source: tests/oracle.rs
3expression: report 3expression: report
4--- 4---
5agreement: 51/58 skeleton lines (87.9%) 5oracle: emacs 31.1, org 9.8.7
6deliberate: 7 line(s) — list-per-bullet-type ×2, pre-code ×2, semantic-emphasis ×2 6agreement: 53/58 skeleton lines (91.4%)
7deliberate: 5 line(s) — list-per-bullet-type ×2, semantic-emphasis ×2
7unexplained: 0 line(s) 8unexplained: 0 line(s)
8(- orgo, + emacs, ~ a difference we mean to have) 9(- orgo, + emacs, ~ a difference we mean to have)
9 10
@@ -60,7 +61,7 @@ unexplained: 0 line(s)
60 "." 61 "."
61 </p> 62 </p>
62 <pre> 63 <pre>
63~ pre-code (1 line(s)) 64 <code>
64 "fn main() { println!(\"hello\"); }" 65 "fn main() { println!(\"hello\"); }"
65~ pre-code (1 line(s)) 66 </code>
66 </pre> 67 </pre>
tests/snapshots/oracle__oracle_elements.snap +5 −4
@@ -2,8 +2,9 @@
2source: tests/oracle.rs 2source: tests/oracle.rs
3expression: report 3expression: report
4--- 4---
5agreement: 64/82 skeleton lines (78.0%) 5oracle: emacs 31.1, org 9.8.7
6deliberate: 26 line(s) — footnote-anchor-naming ×1, footnote-section-shape ×2, id-link-resolution ×1, no-colgroup ×1, pre-code ×2 6agreement: 66/82 skeleton lines (80.5%)
7deliberate: 24 line(s) — footnote-anchor-naming ×1, footnote-section-shape ×2, id-link-resolution ×1, no-colgroup ×1
7unexplained: 0 line(s) 8unexplained: 0 line(s)
8(- orgo, + emacs, ~ a difference we mean to have) 9(- orgo, + emacs, ~ a difference we mean to have)
9 10
@@ -11,9 +12,9 @@ unexplained: 0 line(s)
11 "Code and tables" 12 "Code and tables"
12 </h2> 13 </h2>
13 <pre> 14 <pre>
14~ pre-code (1 line(s)) 15 <code>
15 "fn main() { println!(\"hello\"); }" 16 "fn main() { println!(\"hello\"); }"
16~ pre-code (1 line(s)) 17 </code>
17 </pre> 18 </pre>
18 <table> 19 <table>
19~ no-colgroup (4 line(s)) 20~ no-colgroup (4 line(s))
tests/snapshots/oracle__oracle_footnote.snap +1
@@ -2,6 +2,7 @@
2source: tests/oracle.rs 2source: tests/oracle.rs
3expression: report 3expression: report
4--- 4---
5oracle: emacs 31.1, org 9.8.7
5agreement: 30/53 skeleton lines (56.6%) 6agreement: 30/53 skeleton lines (56.6%)
6deliberate: 29 line(s) — footnote-anchor-naming ×3, footnote-section-shape ×6, semantic-emphasis ×2 7deliberate: 29 line(s) — footnote-anchor-naming ×3, footnote-section-shape ×6, semantic-emphasis ×2
7unexplained: 0 line(s) 8unexplained: 0 line(s)
tests/snapshots/oracle__oracle_headings.snap +4 −3
@@ -2,13 +2,14 @@
2source: tests/oracle.rs 2source: tests/oracle.rs
3expression: report 3expression: report
4--- 4---
5oracle: emacs 31.1, org 9.8.7
5agreement: 28/30 skeleton lines (93.3%) 6agreement: 28/30 skeleton lines (93.3%)
6deliberate: 2 line(s) — semantic-time ×2 7deliberate: 2 line(s) — heading-priority-cookie ×2
7unexplained: 0 line(s) 8unexplained: 0 line(s)
8(- orgo, + emacs, ~ a difference we mean to have) 9(- orgo, + emacs, ~ a difference we mean to have)
9 10
10 <h2> 11 <h2>
11~ semantic-time (1 line(s)) 12~ heading-priority-cookie (1 line(s))
12 </h2> 13 </h2>
13 <p> 14 <p>
14 "A heading carrying a keyword, a priority, tags and a property drawer." 15 "A heading carrying a keyword, a priority, tags and a property drawer."
@@ -20,7 +21,7 @@ unexplained: 0 line(s)
20 "Sub-headings nest by star count." 21 "Sub-headings nest by star count."
21 </p> 22 </p>
22 <h3> 23 <h3>
23~ semantic-time (1 line(s)) 24~ heading-priority-cookie (1 line(s))
24 </h3> 25 </h3>
25 <p> 26 <p>
26 "A priority cookie can stand alone." 27 "A priority cookie can stand alone."
tests/snapshots/oracle__oracle_images.snap +1
@@ -2,6 +2,7 @@
2source: tests/oracle.rs 2source: tests/oracle.rs
3expression: report 3expression: report
4--- 4---
5oracle: emacs 31.1, org 9.8.7
5agreement: 30/42 skeleton lines (71.4%) 6agreement: 30/42 skeleton lines (71.4%)
6deliberate: 14 line(s) — figure-element ×8, semantic-emphasis ×2 7deliberate: 14 line(s) — figure-element ×8, semantic-emphasis ×2
7unexplained: 0 line(s) 8unexplained: 0 line(s)
tests/snapshots/oracle__oracle_lists.snap +1
@@ -2,6 +2,7 @@
2source: tests/oracle.rs 2source: tests/oracle.rs
3expression: report 3expression: report
4--- 4---
5oracle: emacs 31.1, org 9.8.7
5agreement: 109/111 skeleton lines (98.2%) 6agreement: 109/111 skeleton lines (98.2%)
6deliberate: 2 line(s) — semantic-emphasis ×2 7deliberate: 2 line(s) — semantic-emphasis ×2
7unexplained: 0 line(s) 8unexplained: 0 line(s)
tests/snapshots/oracle__oracle_minimal.snap +1
@@ -2,6 +2,7 @@
2source: tests/oracle.rs 2source: tests/oracle.rs
3expression: report 3expression: report
4--- 4---
5oracle: emacs 31.1, org 9.8.7
5agreement: 38/42 skeleton lines (90.5%) 6agreement: 38/42 skeleton lines (90.5%)
6deliberate: 4 line(s) — semantic-emphasis ×4 7deliberate: 4 line(s) — semantic-emphasis ×4
7unexplained: 0 line(s) 8unexplained: 0 line(s)
tests/snapshots/oracle__oracle_table.snap +1
@@ -2,6 +2,7 @@
2source: tests/oracle.rs 2source: tests/oracle.rs
3expression: report 3expression: report
4--- 4---
5oracle: emacs 31.1, org 9.8.7
5agreement: 30/34 skeleton lines (88.2%) 6agreement: 30/34 skeleton lines (88.2%)
6deliberate: 4 line(s) — no-colgroup ×1 7deliberate: 4 line(s) — no-colgroup ×1
7unexplained: 0 line(s) 8unexplained: 0 line(s)
tests/snapshots/oracle__oracle_timestamps.snap +1
@@ -2,6 +2,7 @@
2source: tests/oracle.rs 2source: tests/oracle.rs
3expression: report 3expression: report
4--- 4---
5oracle: emacs 31.1, org 9.8.7
5agreement: 25/62 skeleton lines (40.3%) 6agreement: 25/62 skeleton lines (40.3%)
6deliberate: 37 line(s) — semantic-time ×5 7deliberate: 37 line(s) — semantic-time ×5
7unexplained: 0 line(s) 8unexplained: 0 line(s)