krz/orgo

Lightning fast org-mode static site generator. fast go org-mode static-site-generator

Commit 8844e2b788

8844e2b78830f4a83a77b6e3a56c5780731263c2

parent: 5a02e39f58

Unsigned

cmc <hello@cleberg.net> · 2026-08-11 22:36 UTC

Cut the README down to what a new reader needs

It was 700 lines of pipeline diagrams, phase tables, benchmark numbers and
divergence reports — most of which is architecture notes, and all of which now
has a home on the documentation site. Someone arriving at this repository wants
to know what it makes, how to start, and where the rest is.

So: what it is in three sentences, install, a first site, pointing it at writing
you already have, a table of things you can add when you want them, and a map of
the documentation site. Aimed at someone who writes rather than someone who
builds compilers — no element trees, no hash classes, no oracle.

Everything removed is on https://ccleberg.github.io/orgo/ already, and every
link in the new file was checked against the published site.

Also fixes the install page, which still asked for Rust 1.82 and told people to
clone <repository-url>.

Layout: unified · split

.github/workflows/release.yml +2 −2
@@ -77,7 +77,7 @@ jobs:
77 staging="orgo-${{ github.event.inputs.tag || github.ref_name }}-${{ matrix.target }}" 77 staging="orgo-${{ github.event.inputs.tag || github.ref_name }}-${{ matrix.target }}"
78 mkdir "$staging" 78 mkdir "$staging"
79 cp "target/${{ matrix.target }}/release/orgo" "$staging/" 79 cp "target/${{ matrix.target }}/release/orgo" "$staging/"
80 cp README.org LICENSE CHANGELOG.org "$staging/" 80 cp README.org LICENSE "$staging/"
81 tar czf "$staging.tar.gz" "$staging" 81 tar czf "$staging.tar.gz" "$staging"
82 shasum -a 256 "$staging.tar.gz" > "$staging.tar.gz.sha256" 82 shasum -a 256 "$staging.tar.gz" > "$staging.tar.gz.sha256"
83 83
@@ -101,7 +101,7 @@ jobs:
101 with: 101 with:
102 merge-multiple: true 102 merge-multiple: true
103 103
104 # A draft, deliberately. The changelog entry is written by a person, and a release 104 # A draft, deliberately. The release notes are written by a person, and a release
105 # that publishes itself before anyone has read it cannot be edited quietly. 105 # that publishes itself before anyone has read it cannot be edited quietly.
106 - name: Create the draft release 106 - name: Create the draft release
107 uses: softprops/action-gh-release@3d0d9888cb7fd7b750713d6e236d1fcb99157228 # v3.0.2 107 uses: softprops/action-gh-release@3d0d9888cb7fd7b750713d6e236d1fcb99157228 # v3.0.2
CHANGELOG.org deleted −156
@@ -1,156 +0,0 @@
1* Changelog
2What changed and why, newest first. Entries name the /behaviour/ that moved, since that is
3what a rebuild will show you.
4
5Two conventions worth knowing before reading:
6
7- *A cache-format bump is not a change you need to act on.* The incremental cache is
8 versioned and discards itself; a bump means the next build re-renders everything once.
9- *Output changes are called out.* orgo aims at what Emacs exports from the same
10 file, so an entry that says "now renders X" means your pages will change. That is the
11 product, not a regression — but it belongs in a changelog rather than a diff you find
12 later.
13
14Versions follow the compatibility promise in the README: config keys, template variables,
15CLI flags and URLs are the stable surface.
16
17** 0.19.1
18- Footnote back-links carry =aria-label="Back to reference N"=, and the notes section is
19 labelled. A link whose only visible content is =↩= has that glyph as its whole
20 accessible name, so a screen reader announced "left arrow with hook" once per note with
21 no way to tell them apart.
22
23** 0.19.0
24- *Full-content collections.* =include_content = true= gives a listing template each
25 entry's rendered HTML as =entry.content= — a feed that carries whole posts rather than
26 excerpts. Rendered only when the listing is actually rebuilt, so a cached feed costs
27 nothing.
28- *Fixed: a listing could show a stale excerpt.* Its cache key covered a hand-picked set
29 of fields, and the excerpt was not among them, so rewriting a post's first paragraph
30 left the old text on the index until something unrelated invalidated it. Entries are now
31 hashed through their serialization, which cannot drift from what a template can read.
32 Editing a post's body now rebuilds the listings that show it.
33- =page.toc= entries carry =number=, so a site with section numbering on can number its
34 contents list to match its headings.
35
36** 0.18.0
37Release engineering, so that a version number is worth reading.
38
39- *A written compatibility promise.* Config keys, template variables, CLI flags and URLs
40 are the stable surface; the incremental cache, HTML details and the Rust API are not.
41 In the README, and in the guide under /Versioning and upgrades/.
42- *CI* on Linux and macOS: build, test, clippy as an error, and the documentation site
43 built with =--strict=. Emacs is installed on both, so the oracle suite runs for real
44 instead of skipping.
45- *A checked MSRV*, 1.88 — which is how it came to be 1.88 rather than the 1.82
46 orgo's own code needs. The floor comes from dependencies, and nobody finds that out
47 by reasoning about it.
48- *Release binaries* for macOS (arm64, x86_64) and Linux (gnu, musl), built on tag into
49 a draft release. The tag is checked against =Cargo.toml= before anything is built.
50- A =LICENSE= file to go with the MIT declaration, crates.io metadata, and a release
51 profile that produces a 5.0 MB binary rather than 6.5 MB.
52- This changelog, and =RELEASING.org=.
53
54** 0.17.0
55- *Asset directories outside the source.* =[build] assets = ["../theme/static"]= copies
56 a directory's contents to the site root. A site's static files do not always live where
57 its writing does, and copying them next to the writing is how a repository ends up with
58 two of every stylesheet. =watch= and =serve= watch these directories too. Two files
59 claiming one URL is a build error naming both.
60- *Template hashing is per template.* A page's render key covered every template, so
61 editing a feed template re-rendered the whole site. It now covers the layout the page
62 uses plus what that layout extends, includes or imports. On a 196-page site, editing the
63 feed template renders one page instead of 196.
64- Cache format 7.
65
66** 0.16.0
67- *Org's entity table.* =\alpha=, =\rarr=, =20\deg= and the other 412 names, generated
68 from Emacs' own =org-entities=. An unknown name stays literal; =#+OPTIONS: e:nil= turns
69 the table off. /Output changes/ for any page using entities.
70- *Table captions.* =#+CAPTION:= above a table becomes a numbered =<caption>=.
71- *=#+INCLUDE:= reports itself.* It was inert and silent, which publishes a page with
72 content missing and nobody told. Now a diagnostic, and =--strict= makes it a failure.
73- The Emacs oracle separates deliberate divergence from defects. Every difference from
74 org's exporter is named and justified, and a test asserts there are no others.
75
76** 0.15.0
77Export parity, from a page-by-page diff of a 179-file corpus against the site Emacs
78publishes from the same sources. *All of these change output.*
79
80- Heading levels are relative to a document's shallowest heading, as org exports them.
81- Org's text conversions: =--=, =---=, =...=, and =x^2= / =a_{b}=. Never inside verbatim,
82 code, source blocks or LaTeX. =#+OPTIONS: -:nil=, =^:nil= and =^:{}= all work.
83- Captioned figures are numbered =Figure N:=.
84- A caption attaches to the element /directly/ below it; a blank line between attaches to
85 nothing.
86- Checkboxes render as org writes them, which keeps the =[-]= partly-done state a disabled
87 =<input>= could not express. =[@4]= sets a list item's number.
88- A table's special marker column and its marker rows stay out of the output.
89- =#+BEGIN_NOTE= and any other unrecognised name is a special block: a div holding parsed
90 org rather than a =<pre>= of literal text. Verse keeps its line breaks.
91- Emphasis borders forbid whitespace and nothing else, so =="proxied":false== is verbatim
92 and =~~/.config/doom/config.el~= is a path that starts with a tilde.
93- Listings sort on the time of day when a timestamp carries one.
94- Cache format 6.
95
96** 0.14.0
97- *Per-page layouts.* =[[pages]]= rules map a source path to a template, and
98 =#+TEMPLATE:= on a page overrides any rule. A missing template fails the build naming
99 the page, the template, and what does exist.
100- =page.year=, for grouping a listing by year with minijinja's =groupby=.
101- An explicit nav can order generated pages among authored ones. =nav.mode = "none"= now
102 really means none.
103
104** 0.13.0
105- Bundled TOML and Org syntax definitions, a =syntaxes_dir= for your own, and org's comma
106 escape (=,* heading= inside a block).
107
108** 0.12.0
109- =serve=: a development server with live reload, bound to loopback.
110- A documentation site under =docs/=, built by orgo itself.
111
112** 0.11.0
113- Table of contents as =page.toc=, section numbers, and org's =#+OPTIONS:= per-file
114 switches.
115
116** 0.10.0
117- Excerpts, word count, reading time, a =truncate= filter, and =#+DRAFT:= pages.
118
119** 0.9.0
120- =watch=: rebuilds on OS filesystem events, debounced.
121
122** 0.8.0
123- =site.base_url=, the =absolute= and =rfc822= filters, canonical links, and an RSS feed
124 in the scaffold that validates.
125
126** 0.7.0
127- Pagination for large listings, with a =paginator= template context that composes with
128 grouping.
129
130** 0.6.0
131- Grouped collections: one page per tag plus a tag index.
132- Generated listing pages (=[[collections]]=), sorted indexes, and feeds via XML
133 templates.
134- A config file, user templates, nav modes, an =init= scaffold, and discovery that will
135 not publish =.git=.
136
137** 0.5.0
138- Parse diagnostics carry =file:line=, and pages render in parallel.
139- The corpus audit (=orgo audit=) and the =emacs --batch= oracle.
140- =#+SLUG:= decides a page's output filename — found by auditing a real corpus, where it
141 affected 169 of 182 URLs.
142
143** 0.4.0
144- The full v1 construct scope, with the IN/OUT line under test.
145
146** 0.3.0
147- The incremental build layer: content, config and template hashing, a dependency graph,
148 per-page render keys, and a persisted cache manifest. A full build and an incremental
149 build produce byte-identical output.
150
151** 0.2.0
152- Multi-file site builds: a symbol table, internal link resolution, minijinja templates,
153 tables and footnotes.
154
155** 0.1.0
156- Parse and render a single =.org= file to HTML.
README.org +73 −692
@@ -1,726 +1,107 @@
1* orgo 1* orgo
2An org-mode static site generator, in Rust. Org is treated as the /source language/,
3not an inconvenient input to be normalized into markdown. The org element tree —
4headings, drawers, blocks, links with their org-specific semantics — *is* the
5document model, and we render that tree straight to HTML. We never round-trip through
6a markdown-shaped intermediate representation, because the point is to preserve what
7markdown cannot express: property drawers, TODO/priority/tag metadata on headings,
8=#+= directives, ID links, named/captioned blocks, footnote semantics.
9 2
10The one non-obvious early commitment is *incremental builds keyed on content 3Turn a folder of org files into a website.
11hashing*, treated as a first-class architectural concern from day one. The discipline
12it imposes on the data model — pure, hashable, dependency-tracked units — is the real
13deliverable, even while the corpus is small enough that a full rebuild is instant.
14 4
15*Full documentation is in [[file:docs/][=docs/=]]* — a site written in org and built by 5You write posts the way you already do — an =.org= file per page, in whatever directory
16orgo itself. Build and read it with: 6structure suits you — and orgo builds a complete site from them: pages, navigation, a blog
7index, tags, an RSS feed, syntax-highlighted code. It is one binary with nothing to
8install alongside it, and *you do not need Emacs to build your site*, only to write in a
9format Emacs made.
17 10
18#+begin_src sh 11Org is the source language here, not something to convert away from first. Tools that
19cargo run -- serve docs -o docs/_site 12route org through markdown lose what markdown has no words for — property drawers, a
20#+end_src 13heading's TODO state and tags, =#+= keywords, ID links, captions on images. orgo keeps all
21 14of it, and its output is checked page by page against what Emacs' own exporter produces
22** Quick start 15from the same file.
23#+begin_src sh
24cargo run -- init my-site # config + an editable copy of the layout + a page
25cargo run -- build my-site -o _site
26#+end_src
27
28Or skip the scaffolding entirely — point it at any directory of =.org= files:
29
30#+begin_src sh
31cargo run -- build ~/notes -o _site
32#+end_src
33
34*Zero configuration is a supported path, not a demo.* With no =orgo.toml=, no
35templates and no orgo-specific markup in your files, you get a complete site: pages,
36navigation, syntax-highlighted code and the stylesheet to colour it. Configuration
37changes what you get; it is never what makes it work.
38
39Discovery skips what should not be published — dot-directories such as =.git=, the config
40file, the templates directory, and the output directory when it sits inside the source, so
41=orgo build . -o _site= does the obvious thing.
42
43** Configuration
44Everything is optional. =orgo init= writes a fully commented =orgo.toml=; every
45value below is the default.
46
47#+begin_src toml
48[site]
49title = "orgo site"
50base_url = "" # absolute URL, no trailing slash; needed for feeds/canonical links
51description = ""
52language = "en"
53
54[nav]
55mode = "top-level" # top-level | all | explicit | none
56# pages = ["index.org", "about.org"] # for mode = "explicit"; order is preserved
57
58[templates]
59dir = "templates" # base.html replaces the built-in layout
60expose_page_list = false
61
62# [[pages]] # which layout a section renders through; base.html by default
63# match = "blog" # a source directory or one .org file; most specific rule wins
64# template = "post.html"
65
66[highlight]
67theme = "InspiredGitHub"
68
69[build]
70drafts = false
71assets = [] # extra directories copied to the site root, e.g. ["../theme/static"]
72
73[html]
74heading_offset = 1 # a level-1 org heading becomes <h2>, beneath the layout's <h1>
75#+end_src
76
77*** Templates
78Drop a =base.html= into the templates directory and it replaces the built-in layout
79entirely. Any other =.html= file there is available to ={% include %}= and
80={% extends %}=. Templates are [[https://docs.rs/minijinja][minijinja]] (Jinja2 syntax) and
81receive:
82
83| Variable | What it is |
84|————--+————————————————————————————————————————————————--|
85| =body= | the rendered page HTML — use ={{ body \| safe }}= |
86| =page= | =.title=, =.url=, =.source=, =.date=, =.date_iso=, =.year=, =.tags=, =.content=, =.excerpt=, =.word_count=, =.reading_time=, =.toc=, =.keywords= |
87| =site= | =.title=, =.base_url=, =.description=, =.language= |
88| =nav= | list of ={title, url}=, relative to this page |
89| =root= | =../=-prefix back to the site root from this page |
90| =stylesheet= | URL of the generated =syntax.css= |
91| =pages= | every page's metadata — only when =expose_page_list = true= |
92
93=page.keywords= carries *every* =#+KEYWORD:= in the file under its lowercased name, so
94your own metadata works without this crate knowing about it: =#+CUSTOM_THING: x= is
95={{ page.keywords.custom_thing }}=.
96
97=base.html= is the default layout, not the only one. A =[[pages]]= rule gives a section
98its own — =match = "blog"=, =template = "post.html"= — and =#+TEMPLATE: wide.html= gives
99one page its own, which wins over any rule. A second layout usually starts with
100={% extends "base.html" %}=.
101
102Editing a template re-renders the pages that use it — template sources are a hash input,
103so a design change never leaves a site half-updated.
104
105*** Generated listing pages
106A blog index, an archive, a feed — output files with no source =.org= behind them.
107Repeat the block for each one:
108
109#+begin_src toml
110[[collections]]
111source = "blog" # directory to list; empty means every page
112output = "blog/index.html" # where to write it
113template = "list.html"
114title = "Blog"
115sort = "date" # date | title | path
116order = "desc" # desc | asc
117nav = true # put this listing page in the nav
118#+end_src
119
120The template gets the collection's entries as =pages=, already sorted, plus the usual
121=site=/=nav=/=root=. It can ={% extends "base.html" %}= to inherit the site chrome:
122
123#+begin_src jinja
124{% extends "base.html" %}
125{% block main %}
126<ul>{% for p in pages %}
127 <li><time datetime="{{ p.date_iso }}">{{ p.date_iso }}</time>
128 <a href="{{ root }}{{ p.url }}">{{ p.title }}</a></li>
129{% endfor %}</ul>
130{% endblock %}
131#+end_src
132
133=p.date_iso= is the =YYYY-MM-DD= extracted from =#+DATE:=, whatever org syntax it was
134written in — =[2025-09-05 Fri 10:21:00]=, =<2024-05-01 Wed>= or bare =2024-05-01=. It is
135also the sort key; pages without a parseable date sort last, so an undated draft never
136leads a dated archive.
137
138**** Pagination
139Set =paginate= to split a long listing across numbered pages:
140
141#+begin_src toml
142[[collections]]
143source = "blog"
144output = "blog/index.html"
145paginate = 10
146paginate_output = "blog/page/{n}.html" # {n} is the 1-based page number
147#+end_src
148
149Page 1 stays at =output=, so a section's canonical URL never moves as its page count
150changes; only pages 2..N are named by =paginate_output=. The template gets a =paginator=:
151
152#+begin_src jinja
153{% if paginator and paginator.total > 1 %}
154<nav>
155 {% if paginator.prev_url %}<a href="{{ paginator.prev_url }}">Newer</a>{% endif %}
156 {% for pg in paginator.pages %}
157 <a href="{{ pg.url }}"{% if pg.current %} aria-current="page"{% endif %}>{{ pg.number }}</a>
158 {% endfor %}
159 {% if paginator.next_url %}<a href="{{ paginator.next_url }}">Older</a>{% endif %}
160</nav>
161{% endif %}
162#+end_src
163
164=paginator= carries =current=, =total=, =per_page=, =total_entries=, =prev_url=,
165=next_url=, =first_url=, =last_url=, and =pages=. Every URL is relative to the page
166carrying it, so links work from page 1 (=page/2.html=) and from page 5 (=../index.html=,
167=6.html=) without the template knowing where it sits. An unpaginated collection has no
168=paginator= at all, so ={% if paginator %}= is a reliable test in a shared template.
169
170Grouping and pagination compose: each group paginates independently, which is why
171=paginate_output= needs ={tag}= as well as ={n}= on a grouped collection. An empty
172collection still emits page 1 — a section that exists but has nothing in it should say so
173rather than 404. When the entry count shrinks, pages that no longer exist are deleted
174instead of being left serving stale posts.
175
176**** Tag pages
177Add =group_by= and the collection emits one page /per group/ instead of one page total,
178plus an optional index of the groups:
179
180#+begin_src toml
181[[collections]]
182source = "blog"
183group_by = "tags" # "tags", or any #+KEYWORD: name to group by its value
184output = "tags/{tag}.html" # {tag} is replaced by each group's slug
185template = "tag.html"
186title = "Tagged: {tag}"
187index_output = "tags/index.html" # the tag index
188index_template = "tags.html"
189index_title = "Tags"
190nav = true # adds the *index*, not every tag
191#+end_src
192
193A group page receives its own posts as =pages= and itself as =group=
194(=.name=, =.slug=, =.url=, =.count=). The index receives =groups= — every group, sorted
195by name:
196
197#+begin_src jinja
198<ul>{% for tag in groups %}
199 <li><a href="{{ root }}{{ tag.url }}">{{ tag.name }}</a> ({{ tag.count }})</li>
200{% endfor %}</ul>
201#+end_src
202
203=group_by = "tags"= is multi-valued: a post appears under every tag it carries. Any other
204value names a single-valued =#+KEYWORD:=, so =group_by = "category"= buckets by
205=#+CATEGORY:=.
206
207Two tags that would produce the same URL (=web_dev= and =web@dev= both slugify to
208=web-dev=) are a build error rather than one page silently overwriting the other.
209
210A tag page depends on its own posts and nothing else, so adding a post tagged =rust=
211re-renders that post, its section index, =tags/rust.html=, and the tag index whose counts
212changed — four pages, not one per tag. That precision is why =groups= is given to the
213index and not to every group page: a page that can see every group depends on every
214group.
215
216**** Feeds and absolute URLs
217*A feed is a listing page with an XML template*, not a separate feature — templates are
218loaded by full filename and any extension, so =output = "feed.xml"= with
219=template = "feed.xml"= is all it takes. =orgo init= writes a working RSS template.
220
221A feed is read away from the site that served it, so relative links in one are simply
222broken. Set =site.base_url= and use the =absolute= filter:
223
224#+begin_src jinja
225<link>{{ post.url | absolute }}</link>
226<pubDate>{{ post.date_iso | rfc822 }}</pubDate>
227#+end_src
228 16
229| Filter | Does | 17*Documentation: https://ccleberg.github.io/orgo/* — that site is written in org and built
230|—————+—————————————————————————-| 18by orgo, so it doubles as the longest worked example available.
231| =absolute= | site-root-relative path → absolute URL; already-absolute URLs pass through |
232| =rfc822= | any org or ISO date → the format RSS =pubDate= requires |
233| =truncate(n)= | shorten to at most =n= characters on a word boundary, with an ellipsis |
234 19
235Apply =absolute= to the site-root-relative values — =page.url=, =pages[].url=, 20** Install
236=group.url= — and not to =nav[].url=, =paginator.*_url=, =stylesheet= or =root=, which
237are relative to the page carrying them and already correct there.
238 21
239With no =base_url=, =absolute= is an *error* naming the setting, rather than quietly 22You need [[https://rustup.rs][Rust]] (1.88 or newer). Nothing else — syntax highlighting
240emitting a relative URL that would make the feed invalid everywhere while looking fine. 23and its themes are compiled in.
241The default layout also emits =<link rel="canonical">= when a base URL is set.
242 24
243Listing pages are cached on the entries they list, so adding a post re-renders that 25#+begin_src sh
244section's index and nothing else. 26git clone https://github.com/ccleberg/orgo
245 27cd orgo
246*** Table of contents and =#+OPTIONS:= 28cargo install --path .
247=page.toc= is the page's headings as a *tree* — ={title, anchor, level, children}= —
248because a table of contents is one, and rebuilding a tree from a flat list of levels
249inside a template is what Jinja is worst at. Its anchors come from the same function the
250renderer uses to emit heading =id=s, so a TOC link cannot drift from the heading it
251points at.
252
253#+begin_src jinja
254{% macro toc_list(entries) %}
255<ul>{% for e in entries %}
256 <li><a href="#{{ e.anchor }}">{{ e.title }}</a>
257 {%- if e.children %}{{ toc_list(e.children) }}{% endif %}</li>
258{% endfor %}</ul>
259{% endmacro %}
260{% if page.toc %}{{ toc_list(page.toc) }}{% endif %}
261#+end_src 29#+end_src
262 30
263Org's own per-file export switches are honoured, so a document can turn a feature off for 31That puts an =orgo= command on your =PATH=. Full notes, including how to run it without
264itself the way its author already knows: 32installing anything: https://ccleberg.github.io/orgo/install.html
265
266| Switch | Effect | Site default |
267|———————-+————————————+———————————-|
268| =#+OPTIONS: toc:nil= | empties =page.toc= for this page | =[html] toc = true= |
269| =#+OPTIONS: num:t= | numbers headings =1.=, =1.1.=, … | =[html] section_numbers = false= |
270
271*Section numbers default to off, which differs from Emacs on purpose.*
272=org-export-with-section-numbers= is on there, so an org-published site inherits numbered
273headings whether or not anyone chose them. Most sites do not want them; =num:t= or
274=section_numbers = true= gets Emacs' behaviour back, with Emacs' own
275=section-number-N= classes so the output stays diffable against the oracle.
276
277*** Excerpts and drafts
278=page.excerpt= is a page's =#+DESCRIPTION:= when it sets one and its first paragraph
279otherwise, so a listing has something to show whether or not the author thought about
280summaries. =page.word_count= and =page.reading_time= (minutes at 200 wpm) count prose
281only — a post that is mostly a shell transcript should not read as an hour's work.
282=truncate= exists because an excerpt is usually a whole paragraph and minijinja has no
283such filter.
284
285=#+DRAFT:= keeps a page out of the build entirely — no page, and absent from listings and
286the nav rather than merely unlinked. =--drafts= includes them, which is what you want
287under =watch= while writing one. A draft is out of the symbol table too, so a link /to/
288one is reported as the dead link it would be once published.
289
290The keyword is read forgivingly: =t=, =yes=, =1= and a bare =#+DRAFT:= all mean draft,
291because writing the keyword at all is the signal. Only an explicit =nil=, =false=, =no=,
292=0= or =off= means published.
293
294*** =#+SLUG:=
295A page's output filename comes from its =#+SLUG:= when it has one, so
296=2018-11-28-aes-encryption.org= can publish as =aes-encryption.html=. Without one the
297source filename is used. Slugs are sanitized to a single safe path component, and two
298pages claiming one URL is a build error rather than a silently dropped page.
299
300** Pipeline
301#+begin_example
302DISCOVER → PARSE → INDEX → RESOLVE → RENDER → TEMPLATE → EMIT
303#+end_example
304
305PARSE and RENDER are pure functions of their inputs (cacheable, hashable). INDEX/RESOLVE
306is the only inherently global stage — it is where the link dependency graph is born.
307
308| Stage | Module | Notes |
309|————-+———————-+———————————————————————————-|
310| config | =src/config.rs= | =orgo.toml=: site metadata, nav mode, templates, theme. A hash input. |
311| PARSE | =src/parser.rs= | Hand-written recursive descent: line lexer → element builder → inline tokenizer. |
312| audit | =src/audit.rs= | Phase 0 corpus audit: construct frequencies against the IN/OUT line. |
313| model | =src/model.rs= | The org element tree — Elements (block) vs Objects (inline). |
314| INDEX | =src/index.rs= | Collect link targets into a symbol table. |
315| RESOLVE | =src/resolve.rs= | Rewrite links to URLs; return the used-target list (dependency edges). |
316| RENDER | =src/render.rs= | Tree → HTML fragment; syntect highlighting; footnote two-pass. |
317| TEMPLATE | =src/template.rs= | minijinja: fragment + metadata → full page. |
318| incremental | =src/incremental.rs= | Content/config/template hashing, dep graph, cache manifest, invalidation. |
319
320** v1 scope (delivered as of v0.4; still to be reconciled against a corpus audit)
321*IN — v1 must handle:* headings with nesting, at levels relative to the document's
322shallowest; TODO keywords; priorities =[#A]=; tags; property drawers; plain lists
323(unordered/ordered/description, checkboxes, =[@N]= counters, nesting); tables (with rule
324rows and org's special marker column, no =#+TBLFM:=); source blocks with syntax
325highlighting; example/quote/center/verse blocks and named special blocks; links (external,
326internal =[[*Heading]]=/=[[#custom-id]]=, =id:=); footnotes (inline and referenced); =#+=
327keywords/directives; inline markup (bold/italic/underline/verbatim/code/strike); org's
328export-time text conversions (=--=/=---=/=...=, =x^2=, =a_{b}=, =\alpha=); timestamps
329(active/inactive, ranges); paragraphs and horizontal rules; images with
330=#+CAPTION=/=#+ATTR_HTML=, numbered =Figure N:=.
331 33
332*OUT — explicitly not v1 (parse-and-ignore or reject loudly):* Babel execution / 34** Your first site
333=:results=; =#+TBLFM:= formulas; LaTeX / MathJax (passed through untouched, including past
334the text conversions); =#+INCLUDE:= (never expanded — reported as a diagnostic, so a page
335is never quietly short of content); citations; radio targets and macros; drawers other
336than PROPERTIES/LOGBOOK; column view / clocking / agenda semantics; non-HTML export
337blocks.
338 35
339*Scope guardrail:* every IN item gets a golden-file fixture; every OUT item gets a test
340asserting it degrades predictably (ignored, no crash). The IN/OUT line is enforced by
341=tests/constructs.rs=, defending against the project's #1 risk: scope creep back toward
342all-of-org. Phase 0 checked this line against a real 179-file corpus and found it sound
343(99.9% of construct uses in scope) — but also found one thing missing from it entirely:
344=#+SLUG:=. See [[#phase-0-the-corpus-audit-and-the-emacs-oracle][Phase 0]].
345
346** Phase plan
347| Phase | Scope | Status |
348|——--+——————————————————————————————————————————————————————————+——--|
349| *M0* | *Buildable skeleton: crate layout, module stubs, deps, test harness, fixtures* | *done* |
350| *v0.1* | *End-to-end core parse → render: =build= a single =.org= file to HTML* | *done* |
351| *v0.2* | *Multi-file SITE build: INDEX + RESOLVE internal links, minijinja templates, =build <src-dir> <out-dir>=, tables + footnotes* | *done* |
352| *v0.3* | *Incremental build layer: content/config/template hashing, dependency graph, per-page render keys, persisted cache manifest, invalidation* | *done* |
353| *v0.4* | *MVP: the full v1 construct scope — heading metadata, nested/description lists, block types, timestamps, images, syntect highlighting — with the IN/OUT line under test* | *done* |
354| *0* | *Corpus audit + =emacs --batch= ground-truth oracle* | *done* |
355| 1 | Line lexer + heading/section skeleton | done |
356| 2 | Block elements — lists, source blocks, tables, footnote defs, blocks by type, drawers | done |
357| 3 | Inline objects — emphasis, links, bare URLs, footnote refs, timestamps | done |
358| 4 | Rendering to HTML — tree walk, tables, footnote two-pass, minijinja templating, syntect highlighting | done |
359| 5 | Link resolution + symbol table (INDEX + RESOLVE, used-target list, broken-link reporting) | done |
360| 6 | Incremental build layer (hashing, dep graph, invalidation); =watch= on OS filesystem events | done |
361| *7* | *Hardening: rayon parallelism, error locations in parse diagnostics* | *done* |
362| *8* | *General use: config file, user templates, nav modes, =init= scaffold, safe discovery* | *done* |
363| *9* | *Generated listing pages: =[[collections]]=, sorted indexes, feeds via XML templates* | *done* |
364| *10* | *Grouped collections: one page per tag plus a tag index — full parity with the incumbent* | *done* |
365| *11* | *Pagination: numbered pages with a =paginator= context, composing with grouping* | *done* |
366| *12* | *=base_url=: =absolute=/=rfc822= filters, a valid RSS feed in the scaffold, canonical links* | *done* |
367| *13* | *=watch= on OS filesystem events, debounced, with the feedback loop closed* | *done* |
368| *14* | *Authoring: excerpts, word count, reading time, =truncate=, and draft pages* | *done* |
369| *15* | *Table of contents, section numbers, and org's =#+OPTIONS:= per-file switches* | *done* |
370| *16* | *=serve=: development server with long-poll live reload, loopback-bound* | *done* |
371| *17* | *Bundled TOML and Org syntaxes, a user syntax directory, and org's comma escape* | *done* |
372| *18* | *Per-page layouts: =[[pages]]= rules and =#+TEMPLATE:=* | *done* |
373| *19* | *Export parity: relative heading levels, special strings, sub/superscript, caption numbering, checkbox and counter markup, table marker columns, special blocks* | *done* |
374| *20* | *Correctness debt: org's entity table, table captions, a reported =#+INCLUDE:=, and an oracle that separates deliberate divergence from defects* | *done* |
375| *21* | *Extra asset roots; per-template hashing so one layout edit does not re-render the site* | *done* |
376| *22* | *Release engineering: CI on both platforms, a checked MSRV, release binaries, a changelog, and a written compatibility promise* | *done* |
377
378*** v0.2 in / out
379*Added in v0.2:* the INDEX stage (=SymbolTable= of =:ID:=/=:CUSTOM_ID:=/heading/=file:=
380targets across a directory); the RESOLVE stage — rewrites =[[#custom-id]]=, =[[id:...]]=,
381=[[*Heading]]= and =[[file:other.org]]= links to real relative output URLs, returns the
382=used_targets= list (the =uses= edges, spec §4.3/R2) and reports unresolved links as
383warnings rather than crashing; a minijinja base layout (title, nav, body) applied to every
384page; a =build <src-dir> <out-dir>= path that walks the tree, parses + resolves + renders +
385templates every =.org= into a linked static site and copies non-=.org= assets through;
386plus two new constructs — pipe *tables* (with header band from the rule row) and
387*footnotes* (block =[fn:1]= definitions, referenced =[fn:1]=, and inline =[fn:1:text]=,
388rendered as a numbered, back-linked notes section).
389
390*Left stubbed at v0.2, all closed in v0.4:* timestamps; TODO keywords and priorities;
391generic (non-PROPERTIES) drawers; real syntect tokenizing behind the =Highlighter= trait.
392
393*** v0.3 in / out
394*Added in v0.3 — the incremental build layer (spec §4, the flagship, non-retrofittable
395feature):*
396
397- *Three hash classes (spec §4.1)* in =src/incremental.rs=: a *content hash* (blake3
398 of a file's bytes), a *config hash* (blake3 of the resolved =BuildConfig=), and a
399 *template hash* (blake3 of the template sources). A change in any one invalidates the
400 pages it affects.
401- *Dependency graph (spec §4.3)* built from RESOLVE's =defines=/=uses= edges: a page
402 depends on the targets it links to, so editing (or renaming a heading in) a file
403 invalidates the pages that /link into/ it, not just the file itself — the load-bearing
404 R2 invariant. On rebuild the graph is merged with the previous build's =defines= so a
405 /removed/ target still pulls in its linkers.
406- *Per-page =render_key=* = =H(content ⊕ resolved-links ⊕ config ⊕ template)=. If a
407 page's render key is unchanged, its on-disk output is already correct and it is skipped.
408 The config component folds in a *site-structure hash* (every page's =(path, title)=),
409 because the shared nav bar is global chrome — a title change or a page add/remove alters
410 the nav on every page and so must re-render them all (otherwise byte-equivalence breaks).
411- *Persisted cache manifest* (=<out>/.orgo-cache.json=, JSON), carrying per-page
412 records, the config/template hashes, and the serialized dependency graph, tagged with
413 =CACHE_FORMAT_VERSION=. A version mismatch, a missing file, or a corrupt file all fall
414 back to a clean full rebuild — the cache is an optimization, never a correctness
415 dependency.
416- *Wired into =build_site=*: only pages whose render key changed (or that link into a
417 changed file's targets) are re-rendered; unchanged outputs are left in place. =--no-cache=
418 forces a full rebuild; =clean <out-dir>= removes the output directory (and its cache).
419 =SiteReport= now reports =rendered= vs =skipped= counts.
420
421The hard gates are enforced by =tests/incremental.rs=: full-vs-incremental *byte
422equivalence* (and a second unchanged build re-rendering *zero* pages); *edit-one-file*
423re-renders exactly the changed page plus its linkers; *renamed-heading* invalidates the
424linking page and updates its emitted anchor; and cache *version-bump / missing / corrupt*
425all fall back to a full rebuild.
426
427*Out of scope in v0.3:* real syntect highlighting; timestamps and TODO keywords (all
428landed in v0.4). =watch= is a minimal mtime poll loop (=watch <src-dir> -o <out-dir>=), not
429an OS file-watcher — the fs-notify integration is deferred. The parse-tree cache (spec §4.5,
430"optionally") is not persisted: PARSE/INDEX/RESOLVE run for every file each build (cheap and
431pure); the incremental win is on RENDER + EMIT.
432
433*** v0.4 in / out — the MVP
434v0.4 closes the gap between the v1 scope above and what the code actually did, so every
435construct the IN list claims is now parsed, rendered, and pinned by a golden file:
436
437- *Heading metadata* — TODO keywords (the Emacs default =TODO=/=DONE= set, matched on a
438 word boundary so =TODOs= is not one) and =[#A]= priority cookies, rendered with Emacs'
439 own export classes so the output stays diffable against an =emacs --batch= oracle.
440- *Lists* — indentation-based nesting (a sub-list renders /inside/ its parent =<li>=),
441 multi-paragraph item bodies, and =term :: definition= description lists as =<dl>=.
442- *Blocks by type* — =QUOTE=, =CENTER=, =EXAMPLE=, =EXPORT= and =SRC= are now distinct
443 elements rather than all collapsing to a verbatim example block. Block matching is on the
444 specific kind, so a source block can nest inside a quote. An =html= export block passes
445 through; every other backend drops.
446- *Timestamps* — active =<...>= and inactive =[...]=, optional times, same-day time
447 ranges and =--=-joined date ranges, rendered as =<time>= with a machine-readable
448 =datetime=. Repeater/warning cookies are recognized and discarded.
449- *Images* — a description-less link to an image file renders as =<img>=; with an
450 affiliated =#+CAPTION:=/=#+ATTR_HTML:= it is promoted to a =<figure>= with the caption as
451 both =<figcaption>= and alt text. Links to non-=.org= files are now understood as asset
452 links: neither resolved nor reported as broken.
453- *Syntax highlighting* — real syntect tokenizing to CSS classes (never inline styles, so
454 themes live in the stylesheet). Every build emits the matching =syntax.css= and each page
455 links it relative to its own depth. An unknown language degrades to escaped =<pre><code>=.
456- *Diagnostics* — broken links are reported as the org syntax the author wrote
457 (=warning: b.org: unresolved link [[#setup]]=) rather than a Debug-printed enum.
458
459*The OUT line is now enforced, not just asserted.* =tests/constructs.rs= pins each
460excluded construct to a specific degradation: babel is never executed /and/ a checked-in
461=#+RESULTS:= block is dropped rather than published as if it were verified output;
462=#+TBLFM:= is inert; =#+INCLUDE:= is never expanded and says so; LaTeX, macros and radio targets survive
463as literal text; drawers other than PROPERTIES are captured and dropped; unmodelled block
464types keep their content verbatim.
465
466*Still out:* =#+TODO:= per-file keyword sequences; planning lines
467(=SCHEDULED:=/=DEADLINE:=), which render as ordinary paragraphs; and fixed-width =:=
468lines.
469
470** Serving
471#+begin_src sh 36#+begin_src sh
472cargo run -- serve my-site -o _site # http://127.0.0.1:3000 37orgo init my-site
38orgo serve my-site -o _site
473#+end_src 39#+end_src
474 40
475Builds, watches, serves, and reloads the browser when a rebuild lands — the loop =watch= 41Open http://127.0.0.1:3000. Edit =my-site/index.org=, save, and the page reloads on its
476leaves half-open. 42own — that is the loop you will spend your time in.
477 43
478- *Loopback by default.* A dev server serves unreviewed drafts off your laptop, so 44=init= writes a starter post, a page layout you can edit, and a config file with every
479 reaching the local network is something you ask for with =--host 0.0.0.0=, never 45setting explained in comments. It never overwrites a file you already have.
480 something you get.
481- *The reload script is injected on the way out*, never written to disk. What you
482 deploy is the built site, and it must not carry a dev server's JavaScript.
483- *Long-polling, not WebSockets or SSE.* The browser asks "anything since generation
484 N?" and the server holds the request until there is. Instant like a push, no protocol
485 beyond ordinary HTTP, and no dependency. A streamed response would have been more
486 elegant and does not work: tiny_http buffers a response until its body ends, so a body
487 that never ends never reaches the client.
488- A reload only follows a *successful* rebuild. Reloading onto a stale page because the
489 build just failed tells you nothing; the error is already on your terminal.
490 46
491URL resolution is the server's security boundary and is written as a pure function with 47** Or point it at writing you already have
492its own tests: =..=, percent-encoded =..=, backslashes, absolute paths and embedded NULs
493all resolve to nothing rather than to somewhere outside the output directory.
494 48
495** Watching
496#+begin_src sh 49#+begin_src sh
497cargo run -- watch my-site -o _site 50orgo build ~/notes -o _site
498#+end_src 51#+end_src
499 52
500Rebuilds on OS filesystem events rather than polling, so it costs nothing while nothing 53No config file, no templates, no orgo-specific markup in your files. You get a real site:
501happens. Write bursts are debounced — an editor saving a file writes a temp file, renames 54every page, links between them resolved, navigation across the top, code highlighted. That
502it over the original and touches the directory, which is one edit and several events. 55is a supported way to use it rather than a demo — configuration changes what you get, it
503 56is never what makes it work.
504Two rules decide what counts as a change, and they are not the same rules the build uses
505to find content:
506 57
507- *A build input is a change.* Editing =orgo.toml= or a template rebuilds, even 58Nothing that should stay private is published: dot-directories like =.git=, your templates
508 though discovery skips both as non-content. The question is "would this change the 59and the output folder itself are all skipped.
509 site?", not "is this a page?".
510- *Our own output is not.* =watch . -o _site= puts the output inside the source, so a
511 rebuild's writes raise events that would trigger a rebuild, forever. Dot-directories go
512 the same way — =.git= churns on every command — as do editor scratch files, including
513 Emacs' =file.org~= backups, which do not start with a dot.
514 60
515Where native watching is unavailable (some network and container filesystems), it falls 61Want to know what orgo will make of your files before trusting it with them?
516back to polling and says so, rather than failing. 62=orgo audit ~/notes= reports which org constructs you use and how each one lands, with
63counts and line numbers — never the text of your writing, so the report is safe to share.
517 64
518** Phase 0: the corpus audit and the Emacs oracle 65** What you can add when you want it
519The v1 scope was, by its own admission, /recommended/ — a guess about which slice of org
520matters. Phase 0 replaces both halves of that guess with a measurement: an audit that asks
521what a real corpus actually uses, and an oracle that asks whether we render it the way
522Emacs does.
523 66
524The audit runs against any corpus — point it at your own notes before trusting this tool 67Each of these is a few lines of config, and each has a page in the guide:
525with them. The numbers below come from a 179-file site published today by weblorg, a
526wrapper around org's own HTML exporter, which makes it both a realistic workload and a
527directly comparable incumbent. With collections configured, orgo now reproduces
528*all 182 of that site's URLs*.
529 68
530#+begin_example 69| A blog index, newest first | [[https://ccleberg.github.io/orgo/guide/03-collections.html][Collections]] |
531cargo run -- audit <src-dir> # what does this corpus use, and is it in scope? 70| Tag pages, and an index of tags | [[https://ccleberg.github.io/orgo/guide/03-collections.html][Collections]] |
532cargo test --test oracle # how does our HTML differ from Emacs' own export? 71| An RSS feed | [[https://ccleberg.github.io/orgo/guide/03-collections.html][Collections]] |
533#+end_example 72| Numbered pages when a list gets long | [[https://ccleberg.github.io/orgo/guide/03-collections.html][Collections]] |
73| Your own design, in ordinary HTML templates | [[https://ccleberg.github.io/orgo/guide/04-templates.html][Templates]] |
74| Drafts that stay unpublished until you say so | [[https://ccleberg.github.io/orgo/guide/06-authoring.html][Authoring]] |
75| A table of contents on long posts | [[https://ccleberg.github.io/orgo/guide/06-authoring.html][Authoring]] |
76| Clean URLs that survive a renamed file | [[https://ccleberg.github.io/orgo/guide/06-authoring.html][Authoring]] |
534 77
535*** What the audit found 78Rebuilds only touch the pages that actually changed, so saving a post on a site with
536*The scope guess was sound.* 99.9% of construct uses in the corpus are in scope. The 79hundreds of them stays instant.
537whole out-of-scope tail is 8 uses: four =#+TBLFM:= in a post /about/ org-mode, three
538=\name= entities, and one =#+BEGIN_NOTE=.
539 80
540*=#+SLUG:= was a hole big enough to sink the project.* 178 of 179 files set it, and the 81** The documentation
541published URL comes from it, not from the filename: =2018-11-28-aes-encryption.org= is
542served at =blog/aes-encryption.html=. orgo derived output paths from source filenames,
543so *169 of 179 pages would have been published at the wrong URL* — every inbound link and
544every search result, broken, by a tool that reported a clean build. Output paths now come
545from =#+SLUG:= when present ([[file:src/util.rs][=util::output_path=]]); slugs are sanitized so an
546author-supplied =../../etc/x= cannot escape the output directory, and two pages claiming one
547URL is a build error rather than a silently dropped page. Building the real corpus now
548reproduces all 179 of the live site's URLs exactly.
549 82
550*Some machinery is speculative.* The corpus contains no =id:=, =#custom-id= or =*Heading= 83https://ccleberg.github.io/orgo/
551links at all — its cross-page links are hand-written relative URLs. The INDEX/RESOLVE
552symbol table that v0.2 was built around is, against this corpus, unexercised.
553 84
554*An audit can lie too.* The first run reported 23 uses of a custom TODO keyword sequence. 85| [[https://ccleberg.github.io/orgo/quickstart.html][Quick start]] | A working site in two commands, then your own writing, then your own design. |
555All 23 were false: the detector read the leading word of =* CSS Variables= as the keyword 86| [[https://ccleberg.github.io/orgo/install.html][Install]] | Getting the binary, and running it without installing anything. |
556=CSS=. The corpus defines no =#+TODO:= sequences at all, so the true count was zero. The 87| [[https://ccleberg.github.io/orgo/guide/01-cli.html][Commands]] | Every command and flag, and what each is for. |
557detector now matches conventional keyword names only — a tool that overstates a gap argues 88| [[https://ccleberg.github.io/orgo/guide/02-configuration.html][Configuration]] | Every setting in =orgo.toml=, what it changes, and what it costs. |
558for work nobody needs. 89| [[https://ccleberg.github.io/orgo/guide/03-collections.html][Collections]] | Blog indexes, tag pages, pagination and RSS feeds. |
90| [[https://ccleberg.github.io/orgo/guide/04-templates.html][Templates]] | Layouts, and every variable a template can use. |
91| [[https://ccleberg.github.io/orgo/guide/05-org-support.html][Org support]] | Which org syntax is handled, which is not, and how the rest degrades. |
92| [[https://ccleberg.github.io/orgo/guide/06-authoring.html][Authoring]] | URLs, drafts, excerpts, tables of contents. |
93| [[https://ccleberg.github.io/orgo/guide/07-incremental.html][Incremental builds]] | How it decides what to rebuild. |
94| [[https://ccleberg.github.io/orgo/guide/08-workflow.html][Watching and serving]] | The write-save-see loop. |
95| [[https://ccleberg.github.io/orgo/guide/09-auditing.html][Auditing]] | Reading a corpus before trusting a tool with it. |
96| [[https://ccleberg.github.io/orgo/guide/10-deploying.html][Deploying]] | Producing a production build, and putting it somewhere. |
559 97
560*** What the oracle found 98** Building from a checkout
561=tests/oracle.rs= exports each fixture with org's own exporter via =emacs --batch=, reduces
562both sides to a semantic skeleton (element opens, closes and text, with layout =div=s,
563inline =span=s and all attributes but =href=/=src= dropped), and *snapshots the
564disagreement*. Snapshotting rather than asserting is deliberate: a checked-in divergence
565report gets reviewed and shows up as a diff, where a permanently red test gets ignored.
566Three invariants are asserted outright, and all three hold — heading structure, list
567nesting, and source-block text match Emacs exactly.
568 99
569*No bugs in orgo.* Every remaining divergence is a deliberate choice to emit better 100#+begin_src sh
570HTML than org does: 101cargo test # includes a differential check against Emacs, when present
571 102cargo run -- serve docs -o docs/_site # read the documentation locally
572| | orgo | Emacs | why | 103#+end_src
573|—————--+—————————+—————————-+—————————————|
574| emphasis | =<em>=/=<strong>= | =<i>=/=<b>= | semantic, not presentational |
575| captioned image | =<figure>=/=<figcaption>= | =<p>= + ="Figure 1: …"= | real figure semantics |
576| timestamp | =<time datetime="…">= | literal =<2024-01-15 Mon>= | machine-readable |
577| footnotes | =<section><ol>= | =<h2>Footnotes:</h2>= | a list of notes is a list |
578| heading anchor | slug of the text | =org1a2b3c4= | stable, and what the live site serves |
579| code | =<pre><code>= | =<pre>= | the HTML5 idiom |
580
581One genuine semantic difference: org treats a single blank line between a =1.= list and a
582=-= list as /one/ list and keeps the first item's bullet type, while we start a second list.
583We keep ours, on measurement rather than taste — the pattern occurs *zero* times in the
584corpus, so matching an org quirk would buy nothing and cost the more obvious reading.
585
586*The oracle's best catch was three bugs in itself.* Naive normalization reported code as
587corrupted (it trimmed each of syntect's per-token text runs, turning =def greet= into
588=defgreet=) and reported blocks at 36% agreement (syntect's spans flooded the diff). Both
589were measurement artifacts. A differential harness is a piece of software like any other,
590and the first divergences it reports are usually its own.
591
592** Phase 7: hardening
593*** Parse diagnostics (=file:line: message=)
594The parser's contract is that it always returns a document — out-of-scope and malformed
595constructs degrade rather than crash. The gap was that they degraded /silently/, and in the
596worst cases the degradation is severe: an unterminated =#+BEGIN_SRC= reads the rest of the
597file as block content, and an unterminated drawer does the same but renders to nothing, so
598one missing line deletes most of a page from a build that reports success.
599
600=parse= now returns =Document::diagnostics=, each carrying a 1-based source line, and the
601build prints them as =file:line: message=. =--strict= turns them (and unresolved links) into
602a non-zero exit. Line numbers are threaded as an absolute offset through every nested parse,
603so a block inside a list item inside a section still reports its real file line — there is a
604test for exactly that, because reconstructed and re-indented nested slices are precisely
605where an off-by-N hides. The 179-file corpus produces zero diagnostics.
606
607*** Parallelism
608PARSE, RESOLVE and RENDER/EMIT run under rayon. PARSE is a pure function of one file's bytes
609and RESOLVE only reads the shared symbol table, which is what makes both safe to parallelize
610at all; INDEX stays sequential.
611
612| corpus | before | after | speedup |
613|————————+——--+——-+———|
614| 179 files (real) | 0.23s | 0.07s | 3.3× |
615| 1,790 files (10× copy) | 3.98s | 0.82s | 4.9× |
616
617Measured on 12 cores. =RAYON_NUM_THREADS=1= reproduces the old 3.98s exactly, so the gain is
618parallelism rather than incidental change, and the output is byte-identical to the sequential
619build across the whole corpus.
620
621*Parallelism must not be observable in the result.* =par_iter().collect()= preserves input
622order, so the emitted bytes are unaffected — but the build /report/ is the fragile half:
623pushing to =rendered=/=skipped= from inside the parallel pass would order them by thread
624scheduling, producing a non-deterministic report over a deterministic site. The parallel pass
625therefore returns only what was written, and the report is assembled sequentially afterwards.
626=parallel_builds_are_deterministic_in_output_and_report_order= holds that line, and it was
627verified by reintroducing the bug and watching it fail.
628
629*** The real scaling limit was not the CPU
630Going 10× on corpus size cost 17× in time, which parallelism improves without fixing: the
631cause was the nav bar listing *every* page, so an /n/-page site emitted /n/² nav links. At
6321,790 pages each page carried 1,799 links and the output was 284 MB, against 5.5 MB for the
633179-page corpus — 52× the bytes for 10× the input.
634
635The nav is now built from *top-level pages only* ([[file:src/site.rs][=is_top_level=]]): a nav is a
636map of the site's top level, not an index of its contents, and section pages reach their
637siblings through that section's landing page. Nav size becomes a function of the top level
638rather than of the corpus, and the quadratic disappears.
639
640| 1,790-page corpus (6 top-level pages) | before | after |
641|—————————————+——--+——-|
642| full build | 0.82s | 0.39s |
643| total output | 284 MB | 34 MB |
644| nav links per page | 1,799 | 6 |
645
646Scaling is now linear: 179 pages in 0.07s and 1,796 in 0.39s, where the small case is mostly
647the fixed cost of loading syntect's syntax definitions.
648
649The same rule sharpened the incremental build, which is the larger win. The site-structure
650hash — the thing that forces a global re-render — now covers only the pages that appear in
651the nav, because those are the only ones whose title or URL affects another page. *Adding a
652blog post used to re-render the entire site; now it renders one page.* A top-level page's
653title still invalidates everything, correctly, since every page displays it.
654
655*Trade-off worth knowing:* on a site whose sections live in subdirectories, only genuinely
656root-level pages appear — a site keeping its landing pages at =salary/index.org= and friends
657gets a one-entry nav. That is what =nav.mode = "explicit"= is for: list the pages you want,
658in the order you want them.
659
660*From v0.1 (core subset):* headings with nesting and anchors (every heading is now
661anchored — =:CUSTOM_ID:=/=:ID:= else a slug of its text) and trailing tags; paragraphs;
662plain lists (unordered + ordered) with checkboxes; source blocks; inline markup (=*bold*=,
663=/italic/=, =_underline_=, =+strike+=, ==verbatim==, =~code~=); links and bare URLs.
664
665** Compatibility
666Versions mean something as of 1.0. The *stable surface* — changing incompatibly requires
667a major version — is what you actually build a site against:
668
669| Stable | Detail |
670|——————+——————————————————————————————————————————————-|
671| =orgo.toml= keys | Names, types and meaning. New keys are minor releases; removing one is major. |
672| Template context | =page=, =site=, =nav=, =root=, =pages=, =group=, =groups=, =paginator=, =stylesheet=, and the =absolute= / =rfc822= / =truncate= filters. |
673| CLI | Command names, flags, and exit codes. |
674| URLs | How a source path becomes an output path, including =#+SLUG:=. A generator that moves your URLs breaks every link anyone has to you. |
675
676Explicitly *not stable*, so that the above can be:
677
678- *The incremental cache.* Versioned, discarded on mismatch, never a correctness
679 dependency. It changes whenever it needs to, in any release.
680- *Rendered HTML details.* orgo tracks what Emacs exports from the same file, and
681 closing a gap changes markup. Changes that affect output are called out in
682 [[file:CHANGELOG.org][CHANGELOG.org]] — the class names the documentation names (=post-list=,
683 =figure-number=, =section-number-N=, =footnote-ref=) are the ones to write CSS against.
684- *The Rust API.* The crate is published so the binary can be installed with
685 =cargo install=; the library exists to serve it, and its types move as the tool does.
686
687The *MSRV is 1.88*, checked in CI on every change. orgo's own code compiles on
6881.82; the floor comes from dependencies. Raising it is a minor version, never a patch.
689
690** Dependencies
691Parser is hand-written recursive descent (not =nom=/=chumsky=/=pest= — org is
692line-oriented and context-sensitive, not clean CFG). Key crates: =syntect= (syntax
693highlighting, behind a =Highlighter= trait so tree-sitter can be swapped in later),
694=minijinja= (runtime templates), =blake3= (content/cache hashing), =rayon= (parallel
695PARSE/RESOLVE/RENDER), =notify= (filesystem events for =watch=), =tiny_http= (the =serve=
696development server), =toml= (config), =chrono=, =camino=, =walkdir=, =clap=, =anyhow=/=thiserror=.
697=insta= for snapshot tests, and =emacs --batch= — optional, and only for the oracle.
698
699** Build & test
700#+begin_example
701cargo build
702cargo test # 191 tests
703cargo run -- init my-site # scaffold a new site
704cargo run -- build fixtures/minimal.org -o minimal.html # single file
705cargo run -- build fixtures/site -o _site # whole site (incremental)
706cargo run -- audit fixtures/site # corpus audit (Phase 0)
707cargo run -- build fixtures/site -o _site --no-cache # force a full rebuild
708cargo run -- watch fixtures/site -o _site # rebuild on filesystem events
709cargo run -- serve fixtures/site -o _site # ... and serve with live reload
710cargo run -- clean _site # remove output + cache
711#+end_example
712
713A second =build= of an unchanged site re-renders nothing; editing a page re-renders only
714that page and the pages that link into it (watch the =rendered=/=cached= counts).
715 104
716A build emits =syntax.css= next to its output (the highlighter emits CSS classes, so the 105** Licence
717stylesheet has to come with them) and every page links it.
718 106
719=fixtures/= holds tiny =.org= samples: the core ones (=minimal.org=, =core.org=, 107[[file:LICENSE][0BSD]]. Do what you like with it.
720=elements.org=, =table.org=, =footnote.org=), one per v1 construct group (=headings.org=,
721=lists.org=, =blocks.org=, =timestamps.org=, =images.org=), the scope guardrail
722(=outofscope.org=), and a linked multi-file site under =fixtures/site/= (=index.org=,
723=guide.org=, =about.org= + a =style.css= asset). The real corpus (golden files derived from
724actual documents) lands in Phase 0. =cargo test= runs =insta= snapshots of the element tree
725and rendered HTML for each fixture, the two templated site pages (proving cross-file link
726resolution), and the incremental gates.
RELEASING.org +12 −33
@@ -1,8 +1,6 @@
1* Releasing 1* Releasing
2A release is three things that must agree: a version in =Cargo.toml=, a git tag, and a 2
3changelog entry. The release workflow checks the first two against each other and refuses 3Read the content below for the release process.
4to build if they differ, because a release tagged =v0.18.0= containing a binary that
5reports =0.17.0= is the kind of mistake nobody notices for months.
6 4
7** Before the first publish 5** Before the first publish
8#+begin_src sh 6#+begin_src sh
@@ -11,22 +9,11 @@ cargo publish --dry-run
11#+end_src 9#+end_src
12 10
13=repository= and =homepage= in =Cargo.toml= point at GitHub and at the documentation site 11=repository= and =homepage= in =Cargo.toml= point at GitHub and at the documentation site
14on Pages. If git.krz.sh becomes the primary remote, =repository= should follow it — 12on Pages.
15crates.io shows that link on the crate page, and it should lead somewhere you read.
16 13
17** Every release 14** Every release
181. *Write the changelog entry first.* [[file:CHANGELOG.org][CHANGELOG.org]] names behaviour, not 151. Bump the version in =Cargo.toml=, and build once so =Cargo.lock= follows.
19 commits — someone reading it wants to know what their next build will do differently. 162. Test and package the release.
20 Anything that changes rendered HTML gets said out loud.
21
222. *Bump the version* in =Cargo.toml=, and build once so =Cargo.lock= follows.
23
24 Patch for fixes that change nothing about the stable surface. Minor for new config
25 keys, new template variables, an MSRV bump, or output that changes to track Emacs more
26 closely. Major for anything that breaks the promises in the README's Compatibility
27 section — config keys, template context, CLI, or URLs.
28
293. *Check it.*
30 17
31 #+begin_src sh 18 #+begin_src sh
32 cargo test 19 cargo test
@@ -35,15 +22,9 @@ crates.io shows that link on the crate page, and it should lead somewhere you re
35 cargo package 22 cargo package
36 #+end_src 23 #+end_src
37 24
38 =cargo package= is the one people forget: it builds the crate exactly as crates.io will 253. Build a site you know with =--no-cache= and diff the output against the previous
39 receive it, and catches a file the =exclude= list should not have removed. 26 version's.
40 274. Commit, tag, push.
414. *Verify against a real corpus.* The test suite says the code does what it did; a
42 corpus says the /site/ does. Build a site you know with =--no-cache= and diff the
43 output against the previous version's. A release that quietly changes 200 pages should
44 do so on purpose.
45
465. *Commit, tag, push.*
47 28
48 #+begin_src sh 29 #+begin_src sh
49 git commit -am "0.18: <what changed>" 30 git commit -am "0.18: <what changed>"
@@ -51,7 +32,7 @@ crates.io shows that link on the crate page, and it should lead somewhere you re
51 git push && git push --tags 32 git push && git push --tags
52 #+end_src 33 #+end_src
53 34
546. *Publish the crate.* 355. Publish the crate.
55 36
56 #+begin_src sh 37 #+begin_src sh
57 cargo publish 38 cargo publish
@@ -59,10 +40,8 @@ crates.io shows that link on the crate page, and it should lead somewhere you re
59 40
60 This is irreversible: a published version can be yanked but never replaced. 41 This is irreversible: a published version can be yanked but never replaced.
61 42
627. *Finish the GitHub release.* Pushing the tag builds binaries for macOS (arm64 and 436. Finish the GitHub release. Pushing the tag builds binaries for macOS (arm64 and
63 x86_64) and Linux (gnu and musl) and opens a /draft/ release with them attached. Paste 44 x86_64) and Linux (gnu and musl) and opens a /draft/ release with them attached.
64 the changelog entry in and publish it. The draft is deliberate — a release that
65 publishes itself before anyone has read it cannot be edited quietly.
66 45
67** If a release goes wrong 46** If a release goes wrong
68Yank rather than delete, and ship a fix as a new version: 47Yank rather than delete, and ship a fix as a new version:
@@ -72,4 +51,4 @@ cargo yank --version 0.18.0
72#+end_src 51#+end_src
73 52
74Yanking stops new dependents from selecting it; anyone who already has it keeps working. 53Yanking stops new dependents from selecting it; anyone who already has it keeps working.
75Then release =0.18.1= with the fix and a changelog entry that says what happened. 54Then release =0.18.1= with the fix.
SECURITY.md +1 −1
@@ -11,7 +11,7 @@ Fixes land in a new release rather than as patches to an old one.
11 11
12## Reporting 12## Reporting
13 13
14Email <hello@cleberg.net>, or open a private advisory through GitHub's *Security* tab. 14Email <security@krz.sh>, or open a private advisory through GitHub's *Security* tab.
15Please do not open a public issue for something exploitable. 15Please do not open a public issue for something exploitable.
16 16
17## What is worth reporting 17## What is worth reporting
docs/guide/11-versioning.org +3 −4
@@ -35,8 +35,8 @@ dependency: a missing, stale or corrupt cache produces exactly the same site, mo
35** Rendered HTML details 35** Rendered HTML details
36 36
37orgo aims at what Emacs exports from the same file, and closing a gap changes markup. 37orgo aims at what Emacs exports from the same file, and closing a gap changes markup.
38That is the product working rather than a regression — but it is called out in the 38That is the product working rather than a regression — but it is called out in the release
39changelog every time, because your stylesheet is downstream of it. 39notes every time, because your stylesheet is downstream of it.
40 40
41The class names the documentation names are the ones to write CSS against: 41The class names the documentation names are the ones to write CSS against:
42=post-list=, =post-list-item=, =figure-number=, =table-number=, =section-number-N=, 42=post-list=, =post-list-item=, =figure-number=, =table-number=, =section-number-N=,
@@ -68,5 +68,4 @@ new version's output to the old version's output rather than to a cache written
68mixture of both. =--strict= turns a link that stopped resolving into a failure. 68mixture of both. =--strict= turns a link that stopped resolving into a failure.
69 69
70If you keep your built site in version control, the diff after that command *is* the 70If you keep your built site in version control, the diff after that command *is* the
71upgrade report — which is the most useful review a generator can give you, and the reason 71upgrade report, and the most useful review a generator can give you.
72the changelog names behaviour rather than commits.
docs/install.org +2 −2
@@ -4,7 +4,7 @@
4 4
5* Requirements 5* Requirements
6 6
7- *Rust 1.82 or newer.* Install from [[https://rustup.rs][rustup.rs]] if you do not have it. There is no other 7- *Rust 1.88 or newer.* Install from [[https://rustup.rs][rustup.rs]] if you do not have it. There is no other
8 runtime requirement: the binary is self-contained, with syntax definitions and 8 runtime requirement: the binary is self-contained, with syntax definitions and
9 highlighting themes compiled in. 9 highlighting themes compiled in.
10- *Emacs (optional).* Only the differential test suite uses it, to compare output against 10- *Emacs (optional).* Only the differential test suite uses it, to compare output against
@@ -13,7 +13,7 @@
13* From source 13* From source
14 14
15#+BEGIN_SRC sh 15#+BEGIN_SRC sh
16git clone <repository-url> orgo 16git clone https://github.com/ccleberg/orgo
17cd orgo 17cd orgo
18cargo build --release 18cargo build --release
19#+END_SRC 19#+END_SRC