krz/orgo
Lightning fast org-mode static site generator.
clone: git clone https://gitbay.org/krz/orgo.git
19279bbf18ad26b1591ff81cfc319935e9352194
verified · cmc
author: Christian Cleberg <hello@cleberg.net> · 2026-08-11T04:00:21Z
Cargo.lock | 2 +- Cargo.toml | 2 +- README.md | 89 +++- fixtures/blocks.org | 53 ++ fixtures/headings.org | 20 + fixtures/images.org | 25 + fixtures/lists.org | 38 ++ fixtures/outofscope.org | 55 ++ fixtures/timestamps.org | 21 + src/incremental.rs | 4 +- src/index.rs | 13 + src/main.rs | 16 +- src/model.rs | 8 + src/parser.rs | 586 ++++++++++++++++++--- src/render.rs | 386 ++++++++++++-- src/resolve.rs | 6 + src/site.rs | 24 +- src/template.rs | 10 +- src/util.rs | 10 + tests/constructs.rs | 292 ++++++++++ tests/pipeline.rs | 2 +- tests/site.rs | 2 +- tests/snapshots/constructs__blocks_html.snap | 27 + .../constructs__headings_element_tree.snap | 175 ++++++ tests/snapshots/constructs__headings_html.snap | 14 + tests/snapshots/constructs__images_html.snap | 14 + .../snapshots/constructs__lists_element_tree.snap | 466 ++++++++++++++++ tests/snapshots/constructs__lists_html.snap | 48 ++ tests/snapshots/constructs__out_of_scope_html.snap | 28 + .../constructs__timestamps_element_tree.snap | 209 ++++++++ tests/snapshots/constructs__timestamps_html.snap | 14 + tests/snapshots/pipeline__core_element_tree.snap | 6 + tests/snapshots/pipeline__core_html.snap | 6 +- .../snapshots/pipeline__minimal_element_tree.snap | 18 + tests/snapshots/site__site_guide_html.snap | 1 + tests/snapshots/site__site_index_html.snap | 1 + 36 files changed, 2512 insertions(+), 179 deletions(-) @@ -538,7 +538,7 @@ dependencies = [ [[package]] name = "org-ssg" -version = "0.3.0" +version = "0.4.0" dependencies = [ "anyhow", "blake3", @@ -1,6 +1,6 @@ [package] name = "org-ssg" -version = "0.3.0" +version = "0.4.0" edition = "2021" description = "Org-mode static site generator that renders the org element tree straight to HTML" license = "MIT" @@ -32,7 +32,7 @@ is the only inherently global stage — it is where the link dependency graph is | TEMPLATE | `src/template.rs` | minijinja: fragment + metadata → full page. | | incremental | `src/incremental.rs` | Content/config/template hashing, dep graph, cache manifest, invalidation. | -## v1 scope (recommended; must be reconciled against a corpus audit first) +## v1 scope (delivered as of v0.4; still to be reconciled against a corpus audit) **IN — v1 must handle:** headings with nesting; TODO keywords; priorities `[#A]`; tags; property drawers; plain lists (unordered/ordered/description, checkboxes, nesting); @@ -47,10 +47,11 @@ paragraphs and horizontal rules; images with `#+CAPTION`/`#+ATTR_HTML`. macros; drawers other than PROPERTIES/LOGBOOK; column view / clocking / agenda semantics; non-HTML export blocks; the full Unicode entity set. -**Scope guardrail:** every IN item gets a golden-file fixture from a real document; -every OUT item gets a test asserting it degrades predictably (ignored, no crash). The -IN/OUT line is enforced by tests, defending against the project's #1 risk: scope creep -back toward all-of-org. +**Scope guardrail:** every IN item gets a golden-file fixture; every OUT item gets a test +asserting it degrades predictably (ignored, no crash). The IN/OUT line is enforced by +`tests/constructs.rs`, defending against the project's #1 risk: scope creep back toward +all-of-org. The fixtures are hand-written today; deriving them from a real corpus is +Phase 0. ## Phase plan @@ -60,14 +61,15 @@ back toward all-of-org. | **v0.1** | **End-to-end core parse → render: `build` a single `.org` file to HTML** | **done** | | **v0.2** | **Multi-file SITE build: INDEX + RESOLVE internal links, minijinja templates, `build <src-dir> <out-dir>`, tables + footnotes** | **done** | | **v0.3** | **Incremental build layer: content/config/template hashing, dependency graph, per-page render keys, persisted cache manifest, invalidation** | **done** | +| **v0.4** | **MVP: the full v1 construct scope — heading metadata, nested/description lists, block types, timestamps, images, syntect highlighting — with the IN/OUT line under test** | **done** | | 0 | Corpus audit + `emacs --batch` ground-truth oracle | todo | | 1 | Line lexer + heading/section skeleton | done | -| 2 | Block elements — lists, source blocks, tables, footnote defs done; generic drawers | partial | -| 3 | Inline objects — emphasis, links, bare URLs, footnote refs done; timestamps | partial | -| 4 | Rendering to HTML — tree walk, tables, footnote two-pass, minijinja templating done; real syntect highlighting | partial | +| 2 | Block elements — lists, source blocks, tables, footnote defs, blocks by type, drawers | done | +| 3 | Inline objects — emphasis, links, bare URLs, footnote refs, timestamps | done | +| 4 | Rendering to HTML — tree walk, tables, footnote two-pass, minijinja templating, syntect highlighting | done | | 5 | Link resolution + symbol table (INDEX + RESOLVE, used-target list, broken-link reporting) | done | | 6 | Incremental build layer (hashing, dep graph, invalidation) done; `watch` is a simple poll loop | done | -| 7 | Hardening: rayon parallelism, CLI polish, error locations | todo | +| 7 | Hardening: rayon parallelism, error locations in parse diagnostics | todo | ### v0.2 in / out @@ -82,9 +84,8 @@ plus two new constructs — pipe **tables** (with header band from the rule row) **footnotes** (block `[fn:1]` definitions, referenced `[fn:1]`, and inline `[fn:1:text]`, rendered as a numbered, back-linked notes section). -**Still stubbed (`todo!`):** timestamps; TODO keywords and priorities; generic -(non-PROPERTIES) drawers. Source-block syntax highlighting remains a `<pre><code>` -passthrough behind the `Highlighter` trait; real syntect tokenizing is deferred. +**Left stubbed at v0.2, all closed in v0.4:** timestamps; TODO keywords and priorities; +generic (non-PROPERTIES) drawers; real syntect tokenizing behind the `Highlighter` trait. ### v0.3 in / out @@ -121,12 +122,52 @@ re-renders exactly the changed page plus its linkers; **renamed-heading** invali linking page and updates its emitted anchor; and cache **version-bump / missing / corrupt** all fall back to a full rebuild. -**Out of scope in v0.3 (unchanged from v0.2):** real syntect highlighting; timestamps and -TODO keywords. `watch` is a minimal mtime poll loop (`watch <src-dir> -o <out-dir>`), not an -OS file-watcher — the fs-notify integration is deferred. The parse-tree cache (spec §4.5, +**Out of scope in v0.3:** real syntect highlighting; timestamps and TODO keywords (all +landed in v0.4). `watch` is a minimal mtime poll loop (`watch <src-dir> -o <out-dir>`), not +an OS file-watcher — the fs-notify integration is deferred. The parse-tree cache (spec §4.5, "optionally") is not persisted: PARSE/INDEX/RESOLVE run for every file each build (cheap and pure); the incremental win is on RENDER + EMIT. +### v0.4 in / out — the MVP + +v0.4 closes the gap between the v1 scope above and what the code actually did, so every +construct the IN list claims is now parsed, rendered, and pinned by a golden file: + +- **Heading metadata** — TODO keywords (the Emacs default `TODO`/`DONE` set, matched on a + word boundary so `TODOs` is not one) and `[#A]` priority cookies, rendered with Emacs' + own export classes so the output stays diffable against an `emacs --batch` oracle. +- **Lists** — indentation-based nesting (a sub-list renders *inside* its parent `<li>`), + multi-paragraph item bodies, and `term :: definition` description lists as `<dl>`. +- **Blocks by type** — `QUOTE`, `CENTER`, `EXAMPLE`, `EXPORT` and `SRC` are now distinct + elements rather than all collapsing to a verbatim example block. Block matching is on the + specific kind, so a source block can nest inside a quote. An `html` export block passes + through; every other backend drops. +- **Timestamps** — active `<...>` and inactive `[...]`, optional times, same-day time + ranges and `--`-joined date ranges, rendered as `<time>` with a machine-readable + `datetime`. Repeater/warning cookies are recognized and discarded. +- **Images** — a description-less link to an image file renders as `<img>`; with an + affiliated `#+CAPTION:`/`#+ATTR_HTML:` it is promoted to a `<figure>` with the caption as + both `<figcaption>` and alt text. Links to non-`.org` files are now understood as asset + links: neither resolved nor reported as broken. +- **Syntax highlighting** — real syntect tokenizing to CSS classes (never inline styles, so + themes live in the stylesheet). Every build emits the matching `syntax.css` and each page + links it relative to its own depth. An unknown language degrades to escaped `<pre><code>`. +- **Diagnostics** — broken links are reported as the org syntax the author wrote + (`warning: b.org: unresolved link [[#setup]]`) rather than a Debug-printed enum. + +**The OUT line is now enforced, not just asserted.** `tests/constructs.rs` pins each +excluded construct to a specific degradation: babel is never executed *and* a checked-in +`#+RESULTS:` block is dropped rather than published as if it were verified output; +`#+TBLFM:` is inert; `#+INCLUDE:` is never expanded; LaTeX, macros and radio targets survive +as literal text; drawers other than PROPERTIES are captured and dropped; unmodelled block +types keep their content verbatim. + +**Still out at v0.4:** the Phase 0 corpus audit and `emacs --batch` oracle (the fixtures are +hand-written, so "matches Emacs" is asserted by construction, not measured); rayon +parallelism; parse errors carrying source locations; `#+TODO:` per-file keyword sequences; +planning lines (`SCHEDULED:`/`DEADLINE:`), which render as ordinary paragraphs; fixed-width +`: ` lines; and the `watch` fs-notify integration. + **From v0.1 (core subset):** headings with nesting and anchors (every heading is now anchored — `:CUSTOM_ID:`/`:ID:` else a slug of its text) and trailing tags; paragraphs; plain lists (unordered + ordered) with checkboxes; source blocks; inline markup (`*bold*`, @@ -155,10 +196,14 @@ cargo run -- clean _site # remove output + cach A second `build` of an unchanged site re-renders nothing; editing a page re-renders only that page and the pages that link into it (watch the `rendered`/`cached` counts). -`fixtures/` holds tiny `.org` samples: single-file ones (`minimal.org`, `core.org`, -`elements.org`, `table.org`, `footnote.org`) and a linked multi-file site under -`fixtures/site/` (`index.org`, `guide.org`, `about.org` + a `style.css` asset). The -real corpus (golden files derived from actual documents) lands in Phase 0. `cargo test` -includes `insta` snapshots of the element tree and rendered HTML for the single-file -fixtures, the two templated site pages (proving cross-file link resolution), and the -table and footnote constructs. +A build emits `syntax.css` next to its output (the highlighter emits CSS classes, so the +stylesheet has to come with them) and every page links it. + +`fixtures/` holds tiny `.org` samples: the core ones (`minimal.org`, `core.org`, +`elements.org`, `table.org`, `footnote.org`), one per v1 construct group (`headings.org`, +`lists.org`, `blocks.org`, `timestamps.org`, `images.org`), the scope guardrail +(`outofscope.org`), and a linked multi-file site under `fixtures/site/` (`index.org`, +`guide.org`, `about.org` + a `style.css` asset). The real corpus (golden files derived from +actual documents) lands in Phase 0. `cargo test` runs `insta` snapshots of the element tree +and rendered HTML for each fixture, the two templated site pages (proving cross-file link +resolution), and the incremental gates. new file mode 100644 @@ -0,0 +1,53 @@ +#+TITLE: Blocks + +* Quote + +#+BEGIN_QUOTE +A quoted paragraph with /markup/. + +And a second paragraph. +#+END_QUOTE + +* Center + +#+BEGIN_CENTER +Centred text. +#+END_CENTER + +* Example + +#+BEGIN_EXAMPLE +Verbatim *not bold* text. + Indentation preserved. +#+END_EXAMPLE + +* Export + +#+BEGIN_EXPORT html +<aside class="raw">Raw HTML passes through.</aside> +#+END_EXPORT + +#+BEGIN_EXPORT latex +\emph{A non-HTML backend is dropped.} +#+END_EXPORT + +* Source + +#+BEGIN_SRC python +def greet(name): + return f"hello {name}" +#+END_SRC + +#+BEGIN_SRC +plain block, no language +#+END_SRC + +* Nested + +#+BEGIN_QUOTE +A quote containing a source block: + +#+BEGIN_SRC sh +echo hi +#+END_SRC +#+END_QUOTE new file mode 100644 @@ -0,0 +1,20 @@ +#+TITLE: Heading Metadata + +* TODO [#A] Write the parser :work:rust: +:PROPERTIES: +:CUSTOM_ID: write-parser +:OWNER: nobody +:END: +A heading carrying a keyword, a priority, tags and a property drawer. + +** DONE Nested and finished +Sub-headings nest by star count. + +** [#C] Priority without a keyword +A priority cookie can stand alone. + +* TODOs are not a keyword +The word boundary matters: this heading has no TODO keyword. + +* DONE +A keyword with no title at all. new file mode 100644 @@ -0,0 +1,25 @@ +#+TITLE: Images + +* Bare image + +[[file:diagram.png]] + +* Captioned figure + +#+CAPTION: The pipeline, end to end +#+ATTR_HTML: :width 640 :class diagram +[[file:pipeline.svg]] + +* Caption with markup + +#+CAPTION: A /stylised/ chart +[[file:chart.png]] + +* Quoted attribute values + +#+ATTR_HTML: :alt "a cat, sitting" :loading lazy +[[file:cat.jpg]] + +* Image with a description is a link + +[[file:diagram.png][the diagram]] new file mode 100644 @@ -0,0 +1,38 @@ +#+TITLE: Lists + +* Nesting + +- outer item + - inner item + - deepest item + - second inner +- second outer + +* Ordered + +1. first +2. second + 1. second point one + 2. second point two +3. third + +* Checkboxes + +- [ ] not done +- [X] done +- [-] partially done + +* Description + +- term one :: the first definition +- term two :: the second definition, which is + soft-wrapped across two lines +- /marked up/ term :: definitions hold inline markup + +* Multi-paragraph items + +- an item whose body has two paragraphs + + the second paragraph, indented under the bullet + +- a plain sibling new file mode 100644 @@ -0,0 +1,55 @@ +#+TITLE: Out of Scope +#+INCLUDE: "other.org" + +Every construct here is on the README's explicit OUT list. The contract is not that we +handle them — it is that they degrade predictably and never crash the build. + +* Babel + +#+BEGIN_SRC sh :results output :exports both +echo "the block renders; :results is never executed" +#+END_SRC + +#+RESULTS: +: stale output from a previous evaluation + +* Table formulas + +| item | cost | +|------+------| +| a | 1 | +| b | 2 | +#+TBLFM: $2=vsum(@2..@3) + +* LaTeX + +Inline math $x^2 + y^2$ and a display block: + +\begin{equation} +E = mc^2 +\end{equation} + +* Macros and radio targets + +A macro call {{{author}}} and a <<<radio target>>> stay literal. + +* Drawers + +:LOGBOOK: +CLOCK: [2024-01-15 Mon 09:00]--[2024-01-15 Mon 10:00] => 1:00 +:END: + +:CUSTOM_DRAWER: +Drawer contents are captured and dropped. +:END: + +* Verse + +#+BEGIN_VERSE +An unmodelled block type +keeps its content verbatim. +#+END_VERSE + +* Entities + +The full entity set is out of scope, so \alpha stays literal. new file mode 100644 @@ -0,0 +1,21 @@ +#+TITLE: Timestamps + +* Single + +An active date <2024-01-15 Mon> and an inactive one [2024-01-15 Mon]. + +With a time: <2024-01-15 Mon 10:30>. + +* Ranges + +A same-day time range <2024-01-15 Mon 10:00-11:45>. + +A multi-day range <2024-01-15 Mon>--<2024-01-20 Sat>. + +* Ignored decorations + +A repeater is dropped: <2024-01-15 Mon +1w>. + +* Not timestamps + +Comparisons like 3 < 4 and [not a stamp] stay literal text. @@ -29,7 +29,7 @@ use crate::util::output_url; /// Bump whenever the `Document` type, hashing scheme, or resolution rules change. /// On mismatch: discard cache, full rebuild (spec §4.5). The blake3 crate's major /// version is folded in as the "hash-algo version" so a hash upgrade also busts. -pub const CACHE_FORMAT_VERSION: u32 = 3; +pub const CACHE_FORMAT_VERSION: u32 = 4; /// blake3 hex identity for a content/config/template/render-key hash class (spec §4.1). pub type Hash = ContentHash; @@ -48,7 +48,7 @@ impl Default for BuildConfig { fn default() -> Self { BuildConfig { output_extension: "html".to_string(), - highlighter_theme: "passthrough".to_string(), + highlighter_theme: crate::render::SYNTAX_THEME.to_string(), } } } @@ -21,6 +21,19 @@ pub enum TargetId { File(Utf8PathBuf), } +/// How a target is written in org source, so a broken-link warning names something the +/// author can search for. +impl std::fmt::Display for TargetId { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + match self { + TargetId::Id(s) => write!(f, "[[id:{s}]]"), + TargetId::CustomId(s) => write!(f, "[[#{s}]]"), + TargetId::Heading(s) => write!(f, "[[*{s}]]"), + TargetId::File(p) => write!(f, "[[file:{p}]]"), + } + } +} + impl TargetId { /// A stable string form used to order targets deterministically when hashing /// (so a page's `resolved_links_hash` does not depend on `HashSet` iteration order). @@ -7,13 +7,13 @@ use camino::{Utf8Path, Utf8PathBuf}; use clap::{Parser, Subcommand}; use org_ssg::parser::parse; -use org_ssg::render::{render, Html, SyntectHighlighter}; +use org_ssg::render::{render, syntax_css, Html, SyntectHighlighter}; use org_ssg::resolve::ResolvedDoc; -use org_ssg::site::{build_site, BuildOptions}; +use org_ssg::site::{build_site, BuildOptions, SYNTAX_STYLESHEET}; use org_ssg::template::Templater; #[derive(Parser)] -#[command(name = "org-ssg", about = "Org-mode static site generator")] +#[command(name = "org-ssg", version, about = "Org-mode static site generator")] struct Cli { #[command(subcommand)] command: Command, @@ -153,7 +153,8 @@ fn watch(input: &Utf8Path, output: &Utf8Path) -> Result<()> { /// Single-file build: read → PARSE → RENDER → TEMPLATE → write. No cross-file link /// resolution (there is no corpus to resolve against); links keep their best-effort -/// URLs. Whole-site link resolution lives in [`build_site`]. +/// URLs. Whole-site link resolution lives in [`build_site`]. The syntax stylesheet is +/// written alongside the page, since highlighting emits CSS classes. fn build_file(input: &Utf8Path, output: &Utf8Path) -> Result<()> { let source = fs::read_to_string(input) .with_context(|| format!("reading source file {input}"))?; @@ -168,13 +169,16 @@ fn build_file(input: &Utf8Path, output: &Utf8Path) -> Result<()> { .unwrap_or_else(|| input.file_stem().unwrap_or("untitled").to_string()); let resolved = ResolvedDoc { document }; - let highlighter = SyntectHighlighter; + let highlighter = SyntectHighlighter::new(); let Html(fragment) = render(&resolved, &highlighter); let templater = Templater::new(); let page = templater - .render_page(&title, &fragment, &[]) + .render_page(&title, &fragment, &[], SYNTAX_STYLESHEET) .with_context(|| format!("templating {input}"))?; fs::write(output, page).with_context(|| format!("writing output file {output}"))?; + + let css = output.with_file_name(SYNTAX_STYLESHEET); + fs::write(&css, syntax_css()).with_context(|| format!("writing stylesheet {css}"))?; Ok(()) } @@ -86,6 +86,14 @@ pub enum Element { code: String, }, ExampleBlock(String), + /// An image link carrying affiliated `#+CAPTION:`/`#+ATTR_HTML:` metadata, which + /// promotes it from an inline image to a block-level `<figure>`. + Figure { + link: Link, + caption: Vec<Object>, + /// Raw `#+ATTR_HTML:` attribute string, passed through to the `<img>` tag. + attrs: String, + }, QuoteBlock(Vec<Element>), CenterBlock(Vec<Element>), /// html passes through; others dropped at render (spec §1 OUT). @@ -9,18 +9,24 @@ //! PARSE is a pure function of a single file's bytes (spec §2.1): it never depends on //! another file, which is what makes content-hash caching sound. //! -//! v0.1 scope (the CORE subset): headings + nesting, property drawers on headings, -//! paragraphs, plain lists (unordered + ordered) with checkboxes, source blocks, and -//! inline markup (bold/italic/underline/strike/verbatim/code, links, bare URLs). -//! Out of scope and left graceful (parsed-and-ignored, never crashing): tables, -//! footnotes, timestamps, TODO keywords, non-SRC blocks (kept verbatim as example -//! blocks), generic drawers other than PROPERTIES. +//! Scope is the v1 IN list (README §"v1 scope"): headings with nesting, TODO keywords, +//! priorities, tags and property drawers; paragraphs; plain lists (unordered, ordered, +//! description) with checkboxes and nesting; tables; source/example/quote/center/export +//! blocks; footnotes; `#+` keywords; inline markup, links, timestamps; images with +//! `#+CAPTION`/`#+ATTR_HTML`. +//! +//! Out-of-scope constructs are parsed-and-ignored, never fatal: babel `:results` and +//! `#+TBLFM:` are inert keywords, unknown block types keep their content verbatim as +//! example blocks, generic drawers are captured and dropped at render, and LaTeX, +//! macros and radio targets survive as literal text. use camino::Utf8Path; +use chrono::{NaiveDate, NaiveDateTime, NaiveTime}; use crate::model::{ BlockParams, Bullet, Checkbox, ContentHash, Document, Element, Heading, Keywords, Link, - LinkTarget, List, ListItem, ListKind, Object, Properties, Section, Table, TableRow, + LinkTarget, List, ListItem, ListKind, Object, Properties, Section, Table, TableRow, Timestamp, + TodoKeyword, }; #[derive(Debug, thiserror::Error)] @@ -113,20 +119,23 @@ pub fn parse(path: &Utf8Path, source: &str) -> Result<Document, ParseError> { .collect(); let first = heading_idxs.first().copied().unwrap_or(lines.len()); - // Preamble: document-level keywords are lifted into `keywords`; the remaining - // lines become the root section's block content. + // Preamble: document-level keywords are *copied* into `keywords`, which is the + // metadata map. They are not removed from the body — collecting is not deleting. + // Dropping the lines would merge the paragraphs either side of a keyword and would + // strand affiliated keywords (`#+CAPTION:`) away from the element they belong to; + // left in place, `parse_elements` handles both. Affiliated keywords are not document + // metadata, so they are not copied. { - let mut body: Vec<&str> = Vec::new(); for (l, c) in lines[..first].iter().zip(&classes[..first]) { if *c == Line::Keyword { if let Some((k, v)) = keyword_kv(l) { - keywords.entries.push((k, v)); + if !is_affiliated(&k) { + keywords.entries.push((k, v)); + } } - } else { - body.push(l); } } - root.content = parse_elements(&body); + root.content = parse_elements(&lines[..first]); } // Each heading segment runs from its own line up to (but excluding) the next heading. @@ -206,14 +215,24 @@ fn heading_level(line: &str) -> Option<u8> { } } +/// The default TODO keyword set, matching Emacs' out-of-the-box `org-todo-keywords` +/// (`("TODO" "DONE")`) so our output can be diffed against an `emacs --batch` oracle. +/// Per-file `#+TODO:` sequences are out of scope; the set is a documented [`BuildConfig`] +/// slot for when it becomes configurable. +/// +/// [`BuildConfig`]: crate::incremental::BuildConfig +const TODO_KEYWORDS: &[(&str, bool)] = &[("TODO", false), ("DONE", true)]; + fn parse_heading(line: &str) -> Heading { let level = heading_level(line).unwrap_or(1); let rest = line[level as usize..].trim(); let (title_str, tags) = split_tags(rest); + let (todo, after_todo) = split_todo(title_str.trim()); + let (priority, title_str) = split_priority(after_todo); Heading { level, - todo: None, // TODO keywords: out of scope for v0.1. - priority: None, // priorities: out of scope for v0.1. + todo, + priority, title: inline(title_str.trim()), tags, properties: Properties::default(), @@ -222,6 +241,43 @@ fn parse_heading(line: &str) -> Heading { } } +/// A leading TODO keyword: a bare word from the keyword set, followed by whitespace or +/// end of the heading. `* TODOs are great` is NOT a keyword (no word boundary). +fn split_todo(title: &str) -> (Option<TodoKeyword>, &str) { + let word_end = title.find(char::is_whitespace).unwrap_or(title.len()); + let word = &title[..word_end]; + for (name, done) in TODO_KEYWORDS { + if word == *name { + return ( + Some(TodoKeyword { + name: (*name).to_string(), + done: *done, + }), + title[word_end..].trim_start(), + ); + } + } + (None, title) +} + +/// A priority cookie `[#A]` immediately after the TODO keyword. +fn split_priority(title: &str) -> (Option<char>, &str) { + let Some(rest) = title.strip_prefix("[#") else { + return (None, title); + }; + let mut chars = rest.chars(); + let Some(c) = chars.next().filter(|c| c.is_ascii_alphanumeric()) else { + return (None, title); + }; + match chars.next() { + Some(']') => ( + Some(c.to_ascii_uppercase()), + rest[c.len_utf8() + 1..].trim_start(), + ), + _ => (None, title), + } +} + /// Split a trailing `:tag1:tag2:` cluster off the heading text. fn split_tags(rest: &str) -> (&str, Vec<String>) { let trimmed = rest.trim_end(); @@ -311,6 +367,10 @@ fn parse_property(line: &str) -> Option<(String, String)> { fn parse_elements(lines: &[&str]) -> Vec<Element> { let mut out = Vec::new(); + // Affiliated keywords (`#+CAPTION:` and friends) belong to the element that follows + // them, so they are held aside until that element is built. + let mut affiliated: Vec<(String, String)> = Vec::new(); + let mut drop_next = false; let mut i = 0; while i < lines.len() { let line = lines[i]; @@ -318,70 +378,84 @@ fn parse_elements(lines: &[&str]) -> Vec<Element> { i += 1; continue; } - if let Some((kind, after)) = block_begin(line) { - let mut j = i + 1; - let mut inner = Vec::new(); - while j < lines.len() && !is_block_end(lines[j]) { - inner.push(lines[j]); - j += 1; - } - let code = inner.join("\n"); - if kind.eq_ignore_ascii_case("SRC") { - let (lang, params) = parse_src_header(&after); - out.push(Element::SrcBlock { lang, params, code }); + if let Some((key, value)) = keyword_kv(line) { + if key.eq_ignore_ascii_case("RESULTS") { + // Babel is never executed (README §OUT), so a checked-in `#+RESULTS:` + // block is output from someone else's Emacs session at some other time. + // Emitting it would put unverifiable content on the page dressed as + // real content, so the block it labels is dropped. + drop_next = true; + } else if is_affiliated(&key) { + affiliated.push((key, value)); } else { - // Non-SRC blocks (quote/example/center/export) are kept verbatim for - // v0.1 rather than richly modeled — see module scope note. - out.push(Element::ExampleBlock(code)); + out.push(Element::Keyword { key, value }); } - i = if j < lines.len() { j + 1 } else { j }; - continue; - } - if is_rule(line) { - out.push(Element::HorizontalRule); i += 1; continue; } - if let Some((key, value)) = keyword_kv(line) { - out.push(Element::Keyword { key, value }); - i += 1; - continue; - } - if line.trim_start().starts_with('|') { - let (table, next) = parse_table(lines, i); - out.push(Element::Table(table)); - i = next; - continue; - } - if let Some((label, first_rest)) = footnote_def_label(line) { - let (def, next) = parse_footnote_def(lines, i, label, first_rest); - out.push(def); - i = next; + let (element, next) = parse_one_element(lines, i); + i = next; + if std::mem::take(&mut drop_next) { + affiliated.clear(); continue; } - if is_list_item(line.trim_start()).is_some() { - let (list, next) = parse_list(lines, i); - out.push(Element::List(list)); - i = next; - continue; - } - // Paragraph: gather consecutive soft-wrapped text lines. - let mut para = Vec::new(); - while i < lines.len() { - let l = lines[i]; - if l.trim().is_empty() || is_structural(l) { - break; - } - para.push(l.trim()); - i += 1; - } - if !para.is_empty() { - out.push(Element::Paragraph(inline(¶.join(" ")))); + if let Some(element) = element { + out.push(attach_affiliated(element, std::mem::take(&mut affiliated))); } } out } +/// Build the single element starting at `lines[start]`, returning it with the index of +/// the first line past it. `None` means the lines were consumed without producing an +/// element. `start` is guaranteed non-blank and not an affiliated keyword. +fn parse_one_element(lines: &[&str], start: usize) -> (Option<Element>, usize) { + let line = lines[start]; + if let Some(text) = comment_text(line) { + return (Some(Element::Comment(text)), start + 1); + } + if let Some((kind, after)) = block_begin(line) { + let (el, next) = parse_block(lines, start, &kind, &after); + return (Some(el), next); + } + if let Some(name) = drawer_begin_name(line) { + let (el, next) = parse_drawer(lines, start, name); + return (Some(el), next); + } + if is_rule(line) { + return (Some(Element::HorizontalRule), start + 1); + } + if line.trim_start().starts_with('|') { + let (table, next) = parse_table(lines, start); + return (Some(Element::Table(table)), next); + } + if let Some((label, first_rest)) = footnote_def_label(line) { + let (def, next) = parse_footnote_def(lines, start, label, first_rest); + return (Some(def), next); + } + if is_list_item(line.trim_start()).is_some() { + let (list, next) = parse_list(lines, start); + return (Some(Element::List(list)), next); + } + // Paragraph: gather consecutive soft-wrapped text lines. + let mut para = Vec::new(); + let mut i = start; + while i < lines.len() { + let l = lines[i]; + if l.trim().is_empty() || is_structural(l) { + break; + } + para.push(l.trim()); + i += 1; + } + if para.is_empty() { + // `is_structural` said this line begins a construct that no branch above claimed + // (a stray `#+END_`); skip it rather than looping forever. + return (None, start + 1); + } + (Some(Element::Paragraph(inline(¶.join(" ")))), i) +} + /// Is this line the start of a non-paragraph construct? fn is_structural(line: &str) -> bool { let t = line.trim_start(); @@ -389,12 +463,153 @@ fn is_structural(line: &str) -> bool { || is_block_end(line) || is_rule(line) || keyword_kv(line).is_some() + || comment_text(line).is_some() + || drawer_begin_name(line).is_some() || is_list_item(t).is_some() || t.starts_with('|') || footnote_def_label(line).is_some() || heading_level(line).is_some() } +// --------------------------------------------------------------------------- +// Blocks, drawers, comments, affiliated keywords +// --------------------------------------------------------------------------- + +/// Consume `#+BEGIN_<KIND> … #+END_<KIND>`. Matching is on the *specific* kind so a +/// source block can sit inside a quote block; an unterminated block runs to end of +/// input rather than failing. +fn parse_block(lines: &[&str], start: usize, kind: &str, after: &str) -> (Element, usize) { + let mut inner: Vec<&str> = Vec::new(); + let mut j = start + 1; + while j < lines.len() && !is_block_end_of(lines[j], kind) { + inner.push(lines[j]); + j += 1; + } + let next = if j < lines.len() { j + 1 } else { j }; + let element = match kind.to_ascii_uppercase().as_str() { + "SRC" => { + let (lang, params) = parse_src_header(after); + Element::SrcBlock { + lang, + params, + code: inner.join("\n"), + } + } + "EXAMPLE" => Element::ExampleBlock(inner.join("\n")), + "QUOTE" => Element::QuoteBlock(parse_elements(&inner)), + "CENTER" => Element::CenterBlock(parse_elements(&inner)), + "EXPORT" => Element::ExportBlock { + backend: after.split_whitespace().next().unwrap_or("").to_string(), + raw: inner.join("\n"), + }, + // Out-of-scope block types (verse, comment, ascii, custom) degrade to a verbatim + // example block: content preserved, no crash. + _ => Element::ExampleBlock(inner.join("\n")), + }; + (element, next) +} + +/// `:NAME:` … `:END:` at block level. A PROPERTIES drawer directly under a heading is +/// consumed by [`parse_section_body`]; anything reaching here is a generic drawer, +/// which the renderer drops (README §OUT). +fn parse_drawer(lines: &[&str], start: usize, name: String) -> (Element, usize) { + let mut inner: Vec<&str> = Vec::new(); + let mut j = start + 1; + while j < lines.len() && !lines[j].trim().eq_ignore_ascii_case(":END:") { + inner.push(lines[j]); + j += 1; + } + let next = if j < lines.len() { j + 1 } else { j }; + ( + Element::Drawer { + name, + content: parse_elements(&inner), + }, + next, + ) +} + +/// The drawer name in a `:NAME:` opening line, if this line is one. `:END:` closes a +/// drawer rather than opening one. +fn drawer_begin_name(line: &str) -> Option<String> { + let t = line.trim(); + if !is_drawer_begin(t) { + return None; + } + let name = &t[1..t.len() - 1]; + if name.eq_ignore_ascii_case("END") { + return None; + } + Some(name.to_string()) +} + +/// A comment line: `#` followed by whitespace or nothing. `#+KEY:` is a keyword (checked +/// first) and `#hashtag` is ordinary text. +fn comment_text(line: &str) -> Option<String> { + let rest = line.trim_start().strip_prefix('#')?; + if rest.is_empty() { + return Some(String::new()); + } + if !rest.starts_with(char::is_whitespace) { + return None; + } + Some(rest.trim().to_string()) +} + +/// Keywords that attach to the element that follows them rather than standing alone. +fn is_affiliated(key: &str) -> bool { + let k = key.to_ascii_uppercase(); + matches!(k.as_str(), "CAPTION" | "NAME" | "ATTR_HTML") +} + +/// A paragraph holding nothing but an image link becomes a block-level figure when a +/// `#+CAPTION:`/`#+ATTR_HTML:` precedes it. Affiliated keywords on anything else are +/// parsed and dropped (README §IN covers captions for images only). +fn attach_affiliated(element: Element, affiliated: Vec<(String, String)>) -> Element { + let value = |key: &str| { + affiliated + .iter() + .find(|(k, _)| k.eq_ignore_ascii_case(key)) + .map(|(_, v)| v.clone()) + }; + let caption = value("CAPTION").unwrap_or_default(); + let attrs = value("ATTR_HTML").unwrap_or_default(); + if caption.is_empty() && attrs.is_empty() { + return element; + } + let Element::Paragraph(objs) = &element else { + return element; + }; + let [Object::Link(link)] = objs.as_slice() else { + return element; + }; + if !is_image_target(&link.target) { + return element; + } + Element::Figure { + link: link.clone(), + caption: inline(&caption), + attrs, + } +} + +/// Does this link point at an image file? Drives both figure promotion and inline +/// `<img>` rendering. +pub fn is_image_target(target: &LinkTarget) -> bool { + let path = match target { + LinkTarget::File { path, .. } => path.as_str(), + LinkTarget::External(url) => url.split(['?', '#']).next().unwrap_or(url), + _ => return false, + }; + let Some(ext) = path.rsplit('.').next() else { + return false; + }; + matches!( + ext.to_ascii_lowercase().as_str(), + "png" | "jpg" | "jpeg" | "gif" | "svg" | "webp" | "avif" + ) +} + // --------------------------------------------------------------------------- // Tables (spec §1 IN; `#+TBLFM:` formulas are parse-and-ignored via keyword_kv) // --------------------------------------------------------------------------- @@ -479,39 +694,142 @@ fn parse_footnote_def( (Element::FootnoteDefinition { label, content }, i) } +/// Consume one plain list. Items are delimited by bullets at the list's own indent +/// column; everything indented further is that item's body, re-parsed as block content — +/// which is what makes lists nest. A single blank line does not end a list, but a blank +/// line followed by anything that is not a sibling bullet does. fn parse_list(lines: &[&str], start: usize) -> (List, usize) { - let kind = match is_list_item(lines[start].trim_start()) { - Some(Bullet::Ordered(_)) => ListKind::Ordered, + let base = indent_of(lines[start]); + let family = bullet_family(&is_list_item(lines[start].trim_start()).expect("list item")); + // A list is a description list when its FIRST item carries a `::` term separator. + let kind = match (&family, split_term(item_text(lines[start].trim_start()))) { + (ListKind::Ordered, _) => ListKind::Ordered, + (_, Some(_)) => ListKind::Description, _ => ListKind::Unordered, }; + let mut items = Vec::new(); let mut i = start; - while i < lines.len() { - let t = lines[i].trim_start(); - let bullet = match is_list_item(t) { - Some(b) => b, - None => break, - }; - let item_kind = match bullet { - Bullet::Ordered(_) => ListKind::Ordered, - _ => ListKind::Unordered, + loop { + // Skip blank lines, but only stay in the list if a sibling bullet follows. + let mut j = i; + while j < lines.len() && lines[j].trim().is_empty() { + j += 1; + } + if j >= lines.len() || indent_of(lines[j]) != base { + break; + } + let Some(bullet) = is_list_item(lines[j].trim_start()) else { + break; }; - if item_kind != kind { + if bullet_family(&bullet) != family { break; } - let rest = item_body(t, &bullet); - let (checkbox, text) = split_checkbox(rest); + + // Body = the text after the bullet, plus every following line indented past the + // bullet column (blank lines included, so an item can hold several paragraphs). + let rest = item_body(lines[j].trim_start(), &bullet); + let (checkbox, rest) = split_checkbox(rest); + let (term, rest) = match kind { + ListKind::Description => match split_term(rest) { + Some((term, def)) => (Some(inline(term.trim())), def), + None => (None, rest), + }, + _ => (None, rest), + }; + + let mut body: Vec<String> = vec![rest.trim().to_string()]; + i = j + 1; + while i < lines.len() { + if lines[i].trim().is_empty() { + // Trailing blanks belong to the item only if more of it follows. + let mut k = i; + while k < lines.len() && lines[k].trim().is_empty() { + k += 1; + } + if k < lines.len() && indent_of(lines[k]) > base { + body.resize(body.len() + (k - i), String::new()); + i = k; + continue; + } + break; + } + if indent_of(lines[i]) <= base { + break; + } + body.push(lines[i].to_string()); + i += 1; + } + items.push(ListItem { bullet, checkbox, - term: None, // description lists: out of scope for v0.1. - content: vec![Element::Paragraph(inline(text.trim()))], + term, + content: parse_elements(&dedent(&body)), }); - i += 1; } (List { kind, items }, i) } +/// Ordered and unordered bullets cannot share a list; description items use unordered +/// bullets, so they are the same family. +fn bullet_family(bullet: &Bullet) -> ListKind { + match bullet { + Bullet::Ordered(_) => ListKind::Ordered, + _ => ListKind::Unordered, + } +} + +fn indent_of(line: &str) -> usize { + line.len() - line.trim_start().len() +} + +/// Strip the common leading indent from an item's body lines so the recursive +/// [`parse_elements`] call sees them at column zero. The first entry is already +/// dedented (it is the text that followed the bullet), so it is excluded from the +/// measurement. +fn dedent(body: &[String]) -> Vec<&str> { + let common = body + .iter() + .skip(1) + .filter(|l| !l.trim().is_empty()) + .map(|l| indent_of(l)) + .min() + .unwrap_or(0); + body.iter() + .enumerate() + .map(|(idx, l)| { + if idx == 0 || l.len() < common { + l.as_str() + } else { + &l[common..] + } + }) + .collect() +} + +/// The text of a list item line after its bullet, for kind detection. +fn item_text(t: &str) -> &str { + match is_list_item(t) { + Some(bullet) => item_body(t, &bullet), + None => t, + } +} + +/// Split `term :: definition`. The separator must be surrounded by whitespace (or end +/// the line) so `a::b` in code text is not mistaken for one. +fn split_term(text: &str) -> Option<(&str, &str)> { + let idx = text.find(" :: ").or_else(|| { + text.strip_suffix(" ::") + .map(|before| before.len()) + })?; + let term = &text[..idx]; + if term.trim().is_empty() { + return None; + } + Some((term, text[idx..].trim_start_matches(" ::").trim_start())) +} + /// Text of a list item after its bullet marker. fn item_body<'a>(item: &'a str, bullet: &Bullet) -> &'a str { match bullet { @@ -567,6 +885,15 @@ fn is_block_end(line: &str) -> bool { line.trim_start().to_ascii_uppercase().starts_with("#+END_") } +/// Does this line close a block of exactly `kind`? +fn is_block_end_of(line: &str, kind: &str) -> bool { + let upper = line.trim().to_ascii_uppercase(); + match upper.strip_prefix("#+END_") { + Some(rest) => rest.trim() == kind.to_ascii_uppercase(), + None => false, + } +} + fn parse_src_header(after: &str) -> (Option<String>, BlockParams) { let mut parts = after.splitn(2, char::is_whitespace); let lang = parts.next().filter(|s| !s.is_empty()).map(|s| s.to_string()); @@ -666,6 +993,14 @@ fn parse_inline_run(chars: &[char]) -> Vec<Object> { continue; } } + if c == '<' || c == '[' { + if let Some((obj, next)) = try_timestamp(chars, i) { + flush(&mut buf, &mut out); + out.push(obj); + i = next; + continue; + } + } if is_scheme_start(chars, i) && boundary_before(chars, i) { if let Some((obj, next)) = try_bare_url(chars, i) { flush(&mut buf, &mut out); @@ -818,6 +1153,87 @@ fn try_bare_url(chars: &[char], i: usize) -> Option<(Object, usize)> { )) } +// --------------------------------------------------------------------------- +// Timestamps +// --------------------------------------------------------------------------- + +/// An org timestamp: `<2024-01-15 Mon>` (active) or `[2024-01-15 Mon]` (inactive), with +/// an optional `HH:MM` time, an optional `HH:MM-HH:MM` same-day range, and an optional +/// `--`-joined second stamp for a multi-day range. +fn try_timestamp(chars: &[char], i: usize) -> Option<(Object, usize)> { + let active = chars[i] == '<'; + let (start, same_day_end, has_time, mut next) = parse_stamp(chars, i)?; + let mut end = same_day_end; + if end.is_none() && starts_with_at(chars, next, "--") { + // A range's two halves must agree on activeness, or it is two adjacent stamps. + if chars.get(next + 2) == Some(&chars[i]) { + if let Some((stamp_end, _, _, after)) = parse_stamp(chars, next + 2) { + end = Some(stamp_end); + next = after; + } + } + } + Some(( + Object::Timestamp(Timestamp { + active, + start, + end, + has_time, + }), + next, + )) +} + +/// One bracketed stamp → `(start, same-day end, has_time, index past the bracket)`. +/// Day names (`Mon`) and repeater/warning cookies (`+1w`, `-2d`) are recognized and +/// discarded — they carry no export meaning (README §OUT: agenda semantics). +fn parse_stamp( + chars: &[char], + i: usize, +) -> Option<(NaiveDateTime, Option<NaiveDateTime>, bool, usize)> { + let open = *chars.get(i)?; + let close = match open { + '<' => '>', + '[' => ']', + _ => return None, + }; + let end = (i + 1..chars.len()).find(|&k| chars[k] == close)?; + let body: String = chars[i + 1..end].iter().collect(); + let mut parts = body.split_whitespace(); + let date = NaiveDate::parse_from_str(parts.next()?, "%Y-%m-%d").ok()?; + + let mut has_time = false; + let mut start_time = NaiveTime::MIN; + let mut end_time = None; + for part in parts { + if let Some((from, to)) = parse_time_spec(part) { + has_time = true; + start_time = from; + end_time = to; + } + } + Some(( + date.and_time(start_time), + end_time.map(|t| date.and_time(t)), + has_time, + end + 1, + )) +} + +/// `HH:MM` or `HH:MM-HH:MM`. +fn parse_time_spec(s: &str) -> Option<(NaiveTime, Option<NaiveTime>)> { + let (from, to) = match s.split_once('-') { + Some((a, b)) => (a, Some(b)), + None => (s, None), + }; + let from = NaiveTime::parse_from_str(from, "%H:%M").ok()?; + let to = match to { + Some(b) => Some(NaiveTime::parse_from_str(b, "%H:%M").ok()?), + None => None, + }; + Some((from, to)) +} + fn is_marker(c: char) -> bool { matches!(c, '*' | '/' | '_' | '+' | '=' | '~') } @@ -8,14 +8,23 @@ //! and its cost must be cache-skippable (spec §4.2). Emit CSS classes, not inline //! styles, so themes live in the stylesheet (spec §3.2). //! -//! v0.2 renders: headings (always anchored, with tags), paragraphs, plain lists -//! (unordered/ordered + checkboxes), source/example blocks, horizontal rules, tables -//! (with header band from the rule row), footnotes, and inline markup. Real syntect -//! tokenizing remains a `<pre><code>` passthrough for now (see [`SyntectHighlighter`]). +//! Renders the v1 IN set: headings (always anchored, with TODO keyword, priority and +//! tags), paragraphs, plain lists (unordered/ordered/description, nested, with +//! checkboxes), tables, source blocks (syntect-highlighted), example/quote/center +//! blocks, HTML export blocks, horizontal rules, images and captioned figures, +//! footnotes, timestamps, and inline markup. Out-of-scope elements (generic drawers, +//! comments, stray keywords, non-HTML export blocks) render to nothing. use std::collections::HashMap; +use std::sync::OnceLock; -use crate::model::{Checkbox, Element, LinkTarget, ListKind, Object, Section, TableRow}; +use syntect::highlighting::ThemeSet; +use syntect::html::{css_for_theme_with_class_style, ClassStyle, ClassedHTMLGenerator}; +use syntect::parsing::SyntaxSet; +use syntect::util::LinesWithEndings; + +use crate::model::{Checkbox, Element, Link, LinkTarget, ListKind, Object, Section, TableRow}; +use crate::parser::is_image_target; use crate::resolve::ResolvedDoc; use crate::util::{plain_text, slugify}; @@ -29,25 +38,94 @@ pub trait Highlighter { fn highlight(&self, code: &str, lang: Option<&str>) -> Html; } -/// Default v1 highlighter. For now this is a plain `<pre><code>` passthrough that -/// escapes the code and tags it with a `language-*` class; real syntect tokenizing -/// to CSS-class spans is deferred (spec §3.2, §4.2). -pub struct SyntectHighlighter; +/// The class style used for both the emitted spans and the generated stylesheet. The +/// two must agree or the CSS will not match the markup. +const CLASS_STYLE: ClassStyle = ClassStyle::Spaced; + +/// The syntect theme whose colours become [`syntax_css`]. Mirrored in +/// [`BuildConfig::highlighter_theme`](crate::incremental::BuildConfig) so a theme change +/// flows into the config hash and invalidates every page. +pub const SYNTAX_THEME: &str = "InspiredGitHub"; + +/// Syntect's default syntax definitions, loaded once per process (loading is far more +/// expensive than highlighting, and a site build highlights many blocks). +fn syntax_set() -> &'static SyntaxSet { + static SET: OnceLock<SyntaxSet> = OnceLock::new(); + SET.get_or_init(SyntaxSet::load_defaults_newlines) +} + +/// The stylesheet the emitted highlight classes refer to. Highlighting emits CSS +/// classes rather than inline styles (spec §3.2), so a build must also emit this. +pub fn syntax_css() -> &'static str { + static CSS: OnceLock<String> = OnceLock::new(); + CSS.get_or_init(|| { + let themes = ThemeSet::load_defaults(); + themes + .themes + .get(SYNTAX_THEME) + .and_then(|theme| css_for_theme_with_class_style(theme, CLASS_STYLE).ok()) + .unwrap_or_default() + }) +} + +/// The v1 highlighter: syntect tokenizing to CSS-class spans (spec §3.2, §4.2). A block +/// whose language syntect does not know falls back to escaped `<pre><code>`. +pub struct SyntectHighlighter { + syntaxes: &'static SyntaxSet, +} + +impl SyntectHighlighter { + pub fn new() -> Self { + SyntectHighlighter { + syntaxes: syntax_set(), + } + } +} + +impl Default for SyntectHighlighter { + fn default() -> Self { + Self::new() + } +} impl Highlighter for SyntectHighlighter { fn highlight(&self, code: &str, lang: Option<&str>) -> Html { - let class = match lang { - Some(l) => format!(" class=\"language-{}\"", escape_attr(l)), - None => String::new(), + let Some(syntax) = lang.and_then(|l| self.syntaxes.find_syntax_by_token(l)) else { + return Html(plain_code(code, lang)); }; + let mut generator = + ClassedHTMLGenerator::new_with_class_style(syntax, self.syntaxes, CLASS_STYLE); + for line in LinesWithEndings::from(code) { + if generator + .parse_html_for_line_which_includes_newline(line) + .is_err() + { + return Html(plain_code(code, lang)); + } + } Html(format!( - "<pre><code{}>{}</code></pre>\n", - class, - escape_html(code) + "<pre><code class=\"{} highlight\">{}</code></pre>\n", + language_class(lang), + generator.finalize() )) } } +fn plain_code(code: &str, lang: Option<&str>) -> String { + format!( + "<pre><code class=\"{}\">{}</code></pre>\n", + language_class(lang), + escape_html(code) + ) +} + +fn language_class(lang: Option<&str>) -> String { + match lang { + Some(l) => format!("language-{}", escape_attr(l)), + None => "language-none".to_string(), + } +} + /// Carries the highlighter plus the footnote collector across the tree walk (spec §2.4). struct Renderer<'a> { hl: &'a dyn Highlighter, @@ -91,7 +169,29 @@ impl Renderer<'_> { .clone() .or_else(|| h.id.clone()) .unwrap_or_else(|| slugify(&plain_text(&h.title))); - out.push_str(&format!("<h{} id=\"{}\">", level, escape_attr(&anchor))); + // A heading with no title text has no meaningful slug; emit no `id` at all + // rather than a run of duplicate empty ones. + if anchor.is_empty() { + out.push_str(&format!("<h{}>", level)); + } else { + out.push_str(&format!("<h{} id=\"{}\">", level, escape_attr(&anchor))); + } + // Keyword/priority markup mirrors Emacs' own HTML export classes, so output + // stays diffable against an `emacs --batch` oracle. + if let Some(todo) = &h.todo { + out.push_str(&format!( + "<span class=\"{} {}\">{}</span> ", + if todo.done { "done" } else { "todo" }, + escape_attr(&todo.name), + escape_html(&todo.name) + )); + } + if let Some(priority) = h.priority { + out.push_str(&format!( + "<span class=\"priority\">[#{}]</span> ", + escape_html(&priority.to_string()) + )); + } self.render_objects(&h.title, out); for tag in &h.tags { out.push_str(&format!(" <span class=\"tag\">{}</span>", escape_html(tag))); @@ -113,33 +213,7 @@ impl Renderer<'_> { self.render_objects(objs, out); out.push_str("</p>\n"); } - Element::List(list) => { - let tag = match list.kind { - ListKind::Ordered => "ol", - _ => "ul", - }; - out.push_str(&format!("<{}>\n", tag)); - for item in &list.items { - out.push_str("<li>"); - if let Some(cb) = &item.checkbox { - let checked = matches!(cb, Checkbox::On); - out.push_str(&format!( - "<input type=\"checkbox\" disabled{}> ", - if checked { " checked" } else { "" } - )); - } - match item.content.as_slice() { - [Element::Paragraph(objs)] => self.render_objects(objs, out), - els => { - for el in els { - self.render_element(el, out); - } - } - } - out.push_str("</li>\n"); - } - out.push_str(&format!("</{}>\n", tag)); - } + Element::List(list) => self.render_list(list, out), Element::Table(table) => self.render_table(table, out), Element::SrcBlock { lang, code, .. } => { let Html(h) = self.hl.highlight(code, lang.as_deref()); @@ -148,12 +222,108 @@ impl Renderer<'_> { Element::ExampleBlock(code) => { out.push_str(&format!("<pre>{}</pre>\n", escape_html(code))); } + Element::QuoteBlock(inner) => { + out.push_str("<blockquote>\n"); + for el in inner { + self.render_element(el, out); + } + out.push_str("</blockquote>\n"); + } + Element::CenterBlock(inner) => { + out.push_str("<div class=\"center\">\n"); + for el in inner { + self.render_element(el, out); + } + out.push_str("</div>\n"); + } + // An `html` export block is verbatim output by definition; every other + // backend is out of scope and drops (README §OUT). + Element::ExportBlock { backend, raw } => { + if backend.eq_ignore_ascii_case("html") { + out.push_str(raw); + out.push('\n'); + } + } + Element::Figure { + link, + caption, + attrs, + } => { + out.push_str("<figure>"); + out.push_str(&image_tag(link, attrs, &plain_text(caption))); + if !caption.is_empty() { + out.push_str("<figcaption>"); + self.render_objects(caption, out); + out.push_str("</figcaption>"); + } + out.push_str("</figure>\n"); + } Element::HorizontalRule => out.push_str("<hr>\n"), // Definitions are emitted in the footnotes section, not inline. Element::FootnoteDefinition { .. } => {} - // Out of scope (non-HTML export, generic drawers, stray keywords, comments): - // emitted as nothing rather than crashing. - _ => {} + // Out of scope (generic drawers, stray keywords, comments): emitted as + // nothing rather than crashing. + Element::Drawer { .. } | Element::Keyword { .. } | Element::Comment(_) => {} + } + } + + fn render_list(&mut self, list: &crate::model::List, out: &mut String) { + if list.kind == ListKind::Description { + out.push_str("<dl>\n"); + for item in &list.items { + out.push_str("<dt>"); + if let Some(term) = &item.term { + self.render_objects(term, out); + } + out.push_str("</dt>\n<dd>"); + self.render_item_content(&item.content, out); + out.push_str("</dd>\n"); + } + out.push_str("</dl>\n"); + return; + } + let tag = if list.kind == ListKind::Ordered { + "ol" + } else { + "ul" + }; + out.push_str(&format!("<{}>\n", tag)); + for item in &list.items { + out.push_str("<li>"); + if let Some(cb) = &item.checkbox { + out.push_str(&format!( + "<input type=\"checkbox\" disabled{}> ", + if matches!(cb, Checkbox::On) { + " checked" + } else { + "" + } + )); + } + self.render_item_content(&item.content, out); + out.push_str("</li>\n"); + } + out.push_str(&format!("</{}>\n", tag)); + } + + /// A single-paragraph item renders its text bare — `<li>text<ul>…` rather than + /// `<li><p>text</p><ul>…` — which is what org does and what makes a nested list read + /// as a continuation of its parent item. An item holding *several* paragraphs wraps + /// them all, so they do not run together. + fn render_item_content(&mut self, content: &[Element], out: &mut String) { + let lead_is_bare = matches!(content.first(), Some(Element::Paragraph(_))) + && !content[1..] + .iter() + .any(|el| matches!(el, Element::Paragraph(_))); + let mut rest = content; + if lead_is_bare { + if let Some((Element::Paragraph(objs), tail)) = content.split_first() { + self.render_objects(objs, out); + rest = tail; + } + } + for el in rest { + self.render_element(el, out); } } @@ -224,6 +394,10 @@ impl Renderer<'_> { out.push_str(&format!("<code class=\"verbatim\">{}</code>", escape_html(s))) } Object::Code(s) => out.push_str(&format!("<code>{}</code>", escape_html(s))), + // A description-less link to an image is the image itself, not a link to it. + Object::Link(link) if link.description.is_none() && is_image_target(&link.target) => { + out.push_str(&image_tag(link, "", "")); + } Object::Link(link) => { let href = link_href(&link.target); out.push_str(&format!("<a href=\"{}\">", escape_attr(&href))); @@ -252,8 +426,8 @@ impl Renderer<'_> { )); } Object::LineBreak => out.push_str("<br>\n"), - // Timestamps, entities: out of scope for now. - _ => {} + Object::Timestamp(ts) => out.push_str(×tamp_html(ts)), + Object::Entity(e) => out.push_str(&escape_html(e)), } } @@ -298,6 +472,122 @@ fn collect_defs_in(elements: &[Element], defs: &mut HashMap<String, Vec<Element> } } +/// An `<img>` for an image link, carrying any `#+ATTR_HTML:` attributes and falling back +/// to the caption for alt text — but only when the author did not write an `:alt` of +/// their own, since two `alt` attributes on one tag is invalid HTML. +fn image_tag(link: &Link, attrs: &str, alt: &str) -> String { + let mut pairs = attr_html(attrs); + if !pairs.iter().any(|(k, _)| k.eq_ignore_ascii_case("alt")) { + pairs.insert(0, ("alt".to_string(), alt.to_string())); + } + let attributes: String = pairs + .iter() + .map(|(k, v)| format!(" {}=\"{}\"", escape_attr(k), escape_attr(v))) + .collect(); + format!( + "<img src=\"{}\"{}>", + escape_attr(&link_href(&link.target)), + attributes + ) +} + +/// `#+ATTR_HTML: :width 400 :class hero` → `[(width, 400), (class, hero)]`. Values run to +/// the next `:key` token and may be double-quoted to include spaces. A malformed spec +/// contributes nothing rather than emitting broken markup. +fn attr_html(spec: &str) -> Vec<(String, String)> { + let mut out = Vec::new(); + let mut key: Option<&str> = None; + let mut value = String::new(); + let mut quoted: Option<String> = None; + + let flush = |out: &mut Vec<(String, String)>, key: &mut Option<&str>, value: &mut String| { + if let Some(k) = key.take() { + out.push((k.to_string(), value.trim().to_string())); + } + value.clear(); + }; + + for token in spec.split_whitespace() { + // Inside a quoted value, everything up to the closing quote is literal. + if let Some(buf) = &mut quoted { + buf.push(' '); + buf.push_str(token.trim_end_matches('"')); + if token.ends_with('"') { + value = quoted.take().expect("quoted value in progress"); + } + continue; + } + if let Some(k) = token.strip_prefix(':') { + flush(&mut out, &mut key, &mut value); + if !k.is_empty() { + key = Some(k); + } + continue; + } + if key.is_none() { + continue; + } + if let Some(rest) = token.strip_prefix('"') { + if let Some(inner) = rest.strip_suffix('"') { + value = inner.to_string(); + } else { + quoted = Some(rest.to_string()); + } + continue; + } + if !value.is_empty() { + value.push(' '); + } + value.push_str(token); + } + if let Some(buf) = quoted { + value = buf; + } + flush(&mut out, &mut key, &mut value); + out +} + +/// `<time>` markup for a timestamp. A range emits both endpoints; a same-day range +/// abbreviates its end to just the time. +fn timestamp_html(ts: &crate::model::Timestamp) -> String { + let class = if ts.active { + "timestamp" + } else { + "timestamp inactive" + }; + let one = |dt: &chrono::NaiveDateTime, text: String| { + let attr = if ts.has_time { + dt.format("%Y-%m-%dT%H:%M").to_string() + } else { + dt.format("%Y-%m-%d").to_string() + }; + format!( + "<time class=\"{class}\" datetime=\"{}\">{}</time>", + escape_attr(&attr), + escape_html(&text) + ) + }; + let text_of = |dt: &chrono::NaiveDateTime| { + if ts.has_time { + dt.format("%Y-%m-%d %H:%M").to_string() + } else { + dt.format("%Y-%m-%d").to_string() + } + }; + + let mut out = one(&ts.start, text_of(&ts.start)); + if let Some(end) = &ts.end { + out.push_str("–"); + let text = if end.date() == ts.start.date() && ts.has_time { + end.format("%H:%M").to_string() + } else { + text_of(end) + }; + out.push_str(&one(end, text)); + } + out +} + /// Best-effort URL for a link target. After RESOLVE, internal targets have been /// rewritten to `External` with their final URL; anything still internal here is an /// unresolved link, rendered to a plausible anchor so the page stays self-consistent. @@ -148,6 +148,12 @@ impl Cx<'_> { LinkTarget::Id(id) => TargetId::Id(id.clone()), LinkTarget::Heading(t) => TargetId::Heading(t.clone()), LinkTarget::File { path, .. } => { + // Only `.org` files are pages. A link to an asset (an image, a PDF) is + // already a correct relative URL in the output tree, since assets are + // copied preserving layout — so it is neither resolved nor reported. + if path.extension() != Some("org") { + return; + } TargetId::File(normalize_link_path(self.from, path)) } }; @@ -23,10 +23,10 @@ use crate::incremental::{ use crate::index::{document_targets, SymbolTable, TargetId}; use crate::model::{ContentHash, Document}; use crate::parser::parse; -use crate::render::{render, Html, SyntectHighlighter}; +use crate::render::{render, syntax_css, Html, SyntectHighlighter}; use crate::resolve::resolve; use crate::template::{template_sources, NavItem, Templater}; -use crate::util::output_url; +use crate::util::{output_url, relative_root}; /// A fully built page: source and output paths (relative to their roots) and its /// final templated HTML. @@ -142,7 +142,7 @@ fn prepare_pages(src: &Utf8Path) -> Result<(Vec<PagePrep>, SymbolTable)> { /// touching the output directory. Shared by the tests (full render, every page). pub fn render_site(src: &Utf8Path) -> Result<(Vec<BuiltPage>, BrokenLinks)> { let (preps, _symbols) = prepare_pages(src)?; - let highlighter = SyntectHighlighter; + let highlighter = SyntectHighlighter::new(); let templater = Templater::new(); let mut pages = Vec::new(); @@ -169,11 +169,15 @@ fn render_page( p: &PagePrep, ) -> Result<String> { let Html(fragment) = render(&p.resolved, highlighter); + let stylesheet = format!("{}{}", relative_root(&p.source), SYNTAX_STYLESHEET); templater - .render_page(&p.title, &fragment, &p.nav) + .render_page(&p.title, &fragment, &p.nav, &stylesheet) .with_context(|| format!("templating {}", p.source)) } +/// Site-root-relative name of the generated syntax stylesheet. Every page links to it. +pub const SYNTAX_STYLESHEET: &str = "syntax.css"; + /// Full site build with the incremental layer (spec §4). Renders only the pages whose /// `render_key` changed or that link into a changed file's targets; reuses the on-disk /// output of everything else; persists an updated cache manifest. @@ -243,7 +247,7 @@ pub fn build_site(src: &Utf8Path, out: &Utf8Path, opts: &BuildOptions) -> Result } } - let highlighter = SyntectHighlighter; + let highlighter = SyntectHighlighter::new(); let templater = Templater::new(); let mut report = SiteReport::default(); @@ -267,6 +271,12 @@ pub fn build_site(src: &Utf8Path, out: &Utf8Path, opts: &BuildOptions) -> Result } } + // The syntax stylesheet the highlighter's CSS classes refer to. Written every build + // (it is a few KB and depends only on the theme, which lives in the config hash). + fs::create_dir_all(out).with_context(|| format!("creating {out}"))?; + fs::write(out.join(SYNTAX_STYLESHEET), syntax_css()) + .with_context(|| format!("writing {SYNTAX_STYLESHEET} under {out}"))?; + // Assets are a dumb copy in v0.3 (spec §8 Q11): copy every run. Cheap, and keeps the // full-vs-incremental byte equivalence trivially true for non-`.org` files. for rel in &assets { @@ -295,7 +305,7 @@ pub fn build_site(src: &Utf8Path, out: &Utf8Path, opts: &BuildOptions) -> Result if opts.strict && !report.broken.is_empty() { for (page, target) in &report.broken { - eprintln!("error: unresolved link in {page}: {target:?}"); + eprintln!("error: {page}: unresolved link {target}"); } anyhow::bail!( "{} unresolved internal link(s) under --strict", @@ -303,7 +313,7 @@ pub fn build_site(src: &Utf8Path, out: &Utf8Path, opts: &BuildOptions) -> Result ); } for (page, target) in &report.broken { - eprintln!("warning: unresolved link in {page}: {target:?}"); + eprintln!("warning: {page}: unresolved link {target}"); } Ok(report) @@ -22,6 +22,9 @@ const BASE_TEMPLATE: &str = r#"<!DOCTYPE html> <head> <meta charset="utf-8"> <title>{{ title }}</title> +{%- if stylesheet %} +<link rel="stylesheet" href="{{ stylesheet }}"> +{%- endif %} </head> <body> <nav> @@ -62,18 +65,21 @@ impl Templater { Templater { env } } - /// fragment + page metadata → full HTML page. + /// fragment + page metadata → full HTML page. `stylesheet` is the URL of the + /// syntax-highlighting stylesheet relative to *this* page (highlighting emits CSS + /// classes, so the sheet has to come with it). pub fn render_page( &self, title: &str, body: &str, nav: &[NavItem], + stylesheet: &str, ) -> Result<String, TemplateError> { let tmpl = self .env .get_template("base") .map_err(|e| TemplateError::Render(e.to_string()))?; - tmpl.render(context! { title => title, body => body, nav => nav }) + tmpl.render(context! { title => title, body => body, nav => nav, stylesheet => stylesheet }) .map_err(|e| TemplateError::Render(e.to_string())) } } @@ -78,6 +78,16 @@ pub fn output_url(from_rel: &Utf8Path, to_rel: &Utf8Path, anchor: Option<&str>) } } +/// The `../`-prefix that reaches the site root from the page at `from_rel`. Empty for a +/// top-level page. Used for site-global assets like the syntax stylesheet. +pub fn relative_root(from_rel: &Utf8Path) -> String { + let depth = from_rel + .parent() + .map(|p| p.components().count()) + .unwrap_or(0); + "../".repeat(depth) +} + /// Relative path from `from_dir` to `to`, using `../` where needed. `/`-joined for URLs. fn relative_path(from_dir: &Utf8Path, to: &Utf8Path) -> String { let from_c: Vec<&str> = from_dir.components().map(|c| c.as_str()).collect(); new file mode 100644 @@ -0,0 +1,292 @@ +//! Golden-file coverage of the v1 scope line (README §"v1 scope"). +//! +//! Two halves, and the second is the point: +//! +//! - **IN** — every construct the v1 scope claims gets an element-tree snapshot (parser +//! correctness) and a rendered-HTML snapshot (renderer correctness). +//! - **OUT** — every construct the v1 scope explicitly excludes gets an assertion that it +//! *degrades predictably*: parsed and ignored, content preserved where that is the +//! honest fallback, never a crash and never a half-rendered artifact. +//! +//! The OUT half is the scope guardrail: it is what defends against this project's stated +//! #1 risk, creeping back toward all-of-org. + +use camino::Utf8PathBuf; + +use org_ssg::model::Document; +use org_ssg::parser::parse; +use org_ssg::render::{render, Html, SyntectHighlighter}; +use org_ssg::resolve::ResolvedDoc; + +fn parse_fixture(name: &str) -> Document { + let path = Utf8PathBuf::from(env!("CARGO_MANIFEST_DIR")) + .join("fixtures") + .join(name); + let source = std::fs::read_to_string(&path).expect("read fixture"); + // A stable relative path keeps snapshots free of absolute machine paths. + parse(Utf8PathBuf::from("fixtures").join(name).as_path(), &source).expect("parse fixture") +} + +fn render_fixture(name: &str) -> String { + let document = parse_fixture(name); + let Html(html) = render(&ResolvedDoc { document }, &SyntectHighlighter::new()); + html +} + +// --------------------------------------------------------------------------- +// IN: the constructs v1 promises to handle +// --------------------------------------------------------------------------- + +#[test] +fn headings_element_tree() { + insta::assert_json_snapshot!(parse_fixture("headings.org").root); +} + +#[test] +fn headings_html() { + insta::assert_snapshot!(render_fixture("headings.org")); +} + +#[test] +fn lists_element_tree() { + insta::assert_json_snapshot!(parse_fixture("lists.org").root); +} + +#[test] +fn lists_html() { + insta::assert_snapshot!(render_fixture("lists.org")); +} + +#[test] +fn blocks_html() { + insta::assert_snapshot!(render_fixture("blocks.org")); +} + +#[test] +fn timestamps_element_tree() { + insta::assert_json_snapshot!(parse_fixture("timestamps.org").root); +} + +#[test] +fn timestamps_html() { + insta::assert_snapshot!(render_fixture("timestamps.org")); +} + +#[test] +fn images_html() { + insta::assert_snapshot!(render_fixture("images.org")); +} + +/// A TODO keyword is a whole word from the configured set, not a prefix: `TODOs are not +/// a keyword` is a plain title. This is the boundary rule most likely to regress. +#[test] +fn todo_keyword_requires_a_word_boundary() { + let html = render_fixture("headings.org"); + assert!( + html.contains("<span class=\"todo TODO\">TODO</span> "), + "a real TODO keyword is marked up:\n{html}" + ); + assert!( + !html.contains("<span class=\"todo TODO\">TODO</span> s are"), + "`TODOs` must not be split into a keyword plus a title:\n{html}" + ); +} + +/// Nesting is by indentation, so a nested list must land *inside* its parent `<li>`. +#[test] +fn nested_list_is_nested_in_the_parent_item() { + let html = render_fixture("lists.org"); + assert!( + html.contains("<li>outer item<ul>"), + "an indented sub-list belongs to the item above it:\n{html}" + ); +} + +/// The caption supplies alt text, but an explicit `:alt` must win — emitting both +/// would put two `alt` attributes on one tag. +#[test] +fn explicit_alt_attribute_replaces_the_caption_derived_one() { + let html = render_fixture("images.org"); + assert!( + html.contains("<img src=\"cat.jpg\" alt=\"a cat, sitting\" loading=\"lazy\">"), + "a quoted `:alt` should be the only alt attribute:\n{html}" + ); + for line in html.lines() { + assert!( + line.matches(" alt=").count() <= 1, + "no tag may carry two alt attributes:\n{line}" + ); + } +} + +/// Keywords in the file preamble are *copied* into the metadata map, not removed from +/// the body. Removing them used to merge the paragraphs either side of a keyword and +/// strand `#+CAPTION:` away from the image below it — content damage from a metadata +/// step, in the one region of a file where every real document has keywords. +#[test] +fn preamble_keywords_do_not_disturb_the_content_around_them() { + let source = "#+TITLE: T\n\nOne.\n#+SOMEKEY: v\nTwo.\n\n#+CAPTION: shot\n[[file:a.png]]\n"; + let document = parse(Utf8PathBuf::from("t.org").as_path(), source).expect("parse"); + assert!( + document + .keywords + .entries + .iter() + .any(|(k, v)| k == "TITLE" && v == "T"), + "document metadata is still collected" + ); + let Html(html) = render(&ResolvedDoc { document }, &SyntectHighlighter::new()); + assert!( + html.contains("<p>One.</p>") && html.contains("<p>Two.</p>"), + "a keyword between two paragraphs must not merge them:\n{html}" + ); + assert!( + html.contains("<figcaption>shot</figcaption>"), + "a preamble `#+CAPTION:` must still attach to the image below it:\n{html}" + ); +} + +/// Highlighting must emit CSS classes, never inline styles, so themes live in the +/// stylesheet (spec §3.2) — and the stylesheet the classes refer to must exist. +#[test] +fn highlighting_emits_classes_not_inline_styles() { + let html = render_fixture("blocks.org"); + assert!( + html.contains("<span class=\"storage type function python\">"), + "python source should be tokenized into classed spans:\n{html}" + ); + assert!( + !html.contains("style=\""), + "highlighting must not emit inline styles:\n{html}" + ); + assert!( + org_ssg::render::syntax_css().contains(".storage"), + "the generated stylesheet must define the emitted classes" + ); +} + +/// An unknown language is not an error: the block keeps its content, escaped. +#[test] +fn unknown_source_language_falls_back_to_plain_code() { + let doc = "#+BEGIN_SRC nosuchlang\n<not markup> & such\n#+END_SRC\n"; + let document = parse(Utf8PathBuf::from("t.org").as_path(), doc).expect("parse"); + let Html(html) = render(&ResolvedDoc { document }, &SyntectHighlighter::new()); + assert_eq!( + html, + "<pre><code class=\"language-nosuchlang\"><not markup> & such</code></pre>\n" + ); +} + +// --------------------------------------------------------------------------- +// OUT: the constructs v1 explicitly excludes must degrade, not explode +// --------------------------------------------------------------------------- + +/// The whole OUT fixture parses and renders. This is the crash gate. +#[test] +fn out_of_scope_fixture_renders_without_crashing() { + let html = render_fixture("outofscope.org"); + assert!(!html.is_empty(), "an out-of-scope document still renders"); +} + +#[test] +fn out_of_scope_html() { + insta::assert_snapshot!(render_fixture("outofscope.org")); +} + +/// Babel is never executed and `#+RESULTS:` blocks are never trusted: the source block +/// renders as code, and its stale results do not reach the page. +#[test] +fn babel_is_not_executed_and_results_are_dropped() { + let html = render_fixture("outofscope.org"); + assert!( + html.contains("the block renders; :results is never executed"), + "the source block itself still renders:\n{html}" + ); + assert!( + !html.contains("stale output from a previous evaluation"), + "a `#+RESULTS:` block must not be emitted:\n{html}" + ); +} + +/// `#+TBLFM:` is inert: the table renders with the values as written, and the formula +/// is neither evaluated nor printed. +#[test] +fn table_formulas_are_inert() { + let html = render_fixture("outofscope.org"); + assert!(html.contains("<table>"), "the table still renders:\n{html}"); + assert!( + !html.contains("vsum"), + "the `#+TBLFM:` formula must not reach the page:\n{html}" + ); +} + +/// LaTeX, macros and radio targets have no v1 semantics, so they survive as the literal +/// text the author typed — lossless, and obviously unhandled to a reader. +#[test] +fn latex_macros_and_radio_targets_stay_literal() { + let html = render_fixture("outofscope.org"); + for literal in ["$x^2 + y^2$", "E = mc^2", "{{{author}}}", "\\alpha"] { + assert!( + html.contains(literal), + "`{literal}` should survive as literal text:\n{html}" + ); + } +} + +/// Drawers other than PROPERTIES are captured by the parser and dropped by the +/// renderer — including LOGBOOK clock lines, which are agenda state, not content. +#[test] +fn drawers_are_parsed_and_dropped() { + let html = render_fixture("outofscope.org"); + assert!( + !html.contains("CLOCK:"), + "LOGBOOK contents must not be emitted:\n{html}" + ); + assert!( + !html.contains("Drawer contents are captured and dropped"), + "generic drawer contents must not be emitted:\n{html}" + ); +} + +/// A non-HTML export block is dropped whole: emitting LaTeX into an HTML page would be +/// worse than emitting nothing. +#[test] +fn non_html_export_blocks_are_dropped() { + let html = render_fixture("blocks.org"); + assert!( + html.contains("<aside class=\"raw\">Raw HTML passes through.</aside>"), + "an `html` export block passes through verbatim:\n{html}" + ); + assert!( + !html.contains("\\emph"), + "a `latex` export block must be dropped:\n{html}" + ); +} + +/// An unmodelled block type keeps its content rather than vanishing. +#[test] +fn unknown_block_types_keep_their_content() { + let html = render_fixture("outofscope.org"); + assert!( + html.contains("An unmodelled block type"), + "a verse block degrades to a verbatim example block:\n{html}" + ); +} + +/// `#+INCLUDE:` is not expanded — the build must not silently pull in another file. +#[test] +fn include_is_not_expanded() { + let doc = parse_fixture("outofscope.org"); + assert!( + doc.keywords + .entries + .iter() + .any(|(k, _)| k.eq_ignore_ascii_case("INCLUDE")), + "`#+INCLUDE:` is captured as an inert keyword" + ); + let html = render_fixture("outofscope.org"); + assert!( + !html.contains("other.org"), + "`#+INCLUDE:` must not be expanded or echoed:\n{html}" + ); +} @@ -23,7 +23,7 @@ fn parse_fixture(name: &str) -> Document { fn render_fixture(name: &str) -> String { let document = parse_fixture(name); let resolved = ResolvedDoc { document }; - let Html(html) = render(&resolved, &SyntectHighlighter); + let Html(html) = render(&resolved, &SyntectHighlighter::new()); html } @@ -33,7 +33,7 @@ fn render_fragment(name: &str) -> String { let path = fixtures().join(name); let source = std::fs::read_to_string(&path).expect("read fixture"); let document = parse(Utf8PathBuf::from(name).as_path(), &source).expect("parse"); - let Html(html) = render(&ResolvedDoc { document }, &SyntectHighlighter); + let Html(html) = render(&ResolvedDoc { document }, &SyntectHighlighter::new()); html } new file mode 100644 @@ -0,0 +1,27 @@ +--- +source: tests/constructs.rs +expression: "render_fixture(\"blocks.org\")" +--- +<h1 id="quote">Quote</h1> +<blockquote> +<p>A quoted paragraph with <em>markup</em>.</p> +<p>And a second paragraph.</p> +</blockquote> +<h1 id="center">Center</h1> +<div class="center"> +<p>Centred text.</p> +</div> +<h1 id="example">Example</h1> +<pre>Verbatim *not bold* text. + Indentation preserved.</pre> +<h1 id="export">Export</h1> +<aside class="raw">Raw HTML passes through.</aside> +<h1 id="source">Source</h1> +<pre><code class="language-python highlight"><span class="source python"><span class="meta function python"><span class="storage type function python">def</span> <span class="entity name function python"><span class="meta generic-name python">greet</span></span></span><span class="meta function parameters python"><span class="punctuation section parameters begin python">(</span></span><span class="meta function parameters python"><span class="variable parameter python">name</span><span class="punctuation section parameters end python">)</span></span><span class="meta function python"><span class="punctuation section function begin python">:</span></span> + <span class="keyword control flow return python">return</span> <span class="storage type string python">f</span><span class="meta string interpolated python"><span class="string quoted double python"><span class="punctuation definition string begin python">"</span></span></span><span class="meta string interpolated python"><span class="string quoted double python">hello </span><span class="meta interpolation python"><span class="punctuation section interpolation begin python">{</span><span class="source python embedded"><span class="meta qualified-name python"><span class="meta generic-name python">name</span></span></span></span><span class="meta interpolation python"><span class="punctuation section interpolation end python">}</span></span><span class="string quoted double python"><span class="punctuation definition string end python">"</span></span></span></span></code></pre> +<pre><code class="language-none">plain block, no language</code></pre> +<h1 id="nested">Nested</h1> +<blockquote> +<p>A quote containing a source block:</p> +<pre><code class="language-sh highlight"><span class="source shell bash"><span class="meta function-call shell"><span class="support function echo shell">echo</span></span><span class="meta function-call arguments shell"> hi</span></span></code></pre> +</blockquote> new file mode 100644 @@ -0,0 +1,175 @@ +--- +source: tests/constructs.rs +expression: "parse_fixture(\"headings.org\").root" +--- +{ + "heading": null, + "content": [ + { + "Keyword": { + "key": "TITLE", + "value": "Heading Metadata" + } + } + ], + "children": [ + { + "heading": { + "level": 1, + "todo": { + "name": "TODO", + "done": false + }, + "priority": "A", + "title": [ + { + "Text": "Write the parser" + } + ], + "tags": [ + "work", + "rust" + ], + "properties": { + "entries": [ + [ + "CUSTOM_ID", + "write-parser" + ], + [ + "OWNER", + "nobody" + ] + ] + }, + "id": null, + "custom_id": "write-parser" + }, + "content": [ + { + "Paragraph": [ + { + "Text": "A heading carrying a keyword, a priority, tags and a property drawer." + } + ] + } + ], + "children": [ + { + "heading": { + "level": 2, + "todo": { + "name": "DONE", + "done": true + }, + "priority": null, + "title": [ + { + "Text": "Nested and finished" + } + ], + "tags": [], + "properties": { + "entries": [] + }, + "id": null, + "custom_id": null + }, + "content": [ + { + "Paragraph": [ + { + "Text": "Sub-headings nest by star count." + } + ] + } + ], + "children": [] + }, + { + "heading": { + "level": 2, + "todo": null, + "priority": "C", + "title": [ + { + "Text": "Priority without a keyword" + } + ], + "tags": [], + "properties": { + "entries": [] + }, + "id": null, + "custom_id": null + }, + "content": [ + { + "Paragraph": [ + { + "Text": "A priority cookie can stand alone." + } + ] + } + ], + "children": [] + } + ] + }, + { + "heading": { + "level": 1, + "todo": null, + "priority": null, + "title": [ + { + "Text": "TODOs are not a keyword" + } + ], + "tags": [], + "properties": { + "entries": [] + }, + "id": null, + "custom_id": null + }, + "content": [ + { + "Paragraph": [ + { + "Text": "The word boundary matters: this heading has no TODO keyword." + } + ] + } + ], + "children": [] + }, + { + "heading": { + "level": 1, + "todo": { + "name": "DONE", + "done": true + }, + "priority": null, + "title": [], + "tags": [], + "properties": { + "entries": [] + }, + "id": null, + "custom_id": null + }, + "content": [ + { + "Paragraph": [ + { + "Text": "A keyword with no title at all." + } + ] + } + ], + "children": [] + } + ] +} new file mode 100644 @@ -0,0 +1,14 @@ +--- +source: tests/constructs.rs +expression: "render_fixture(\"headings.org\")" +--- +<h1 id="write-parser"><span class="todo TODO">TODO</span> <span class="priority">[#A]</span> Write the parser <span class="tag">work</span> <span class="tag">rust</span></h1> +<p>A heading carrying a keyword, a priority, tags and a property drawer.</p> +<h2 id="nested-and-finished"><span class="done DONE">DONE</span> Nested and finished</h2> +<p>Sub-headings nest by star count.</p> +<h2 id="priority-without-a-keyword"><span class="priority">[#C]</span> Priority without a keyword</h2> +<p>A priority cookie can stand alone.</p> +<h1 id="todos-are-not-a-keyword">TODOs are not a keyword</h1> +<p>The word boundary matters: this heading has no TODO keyword.</p> +<h1><span class="done DONE">DONE</span> </h1> +<p>A keyword with no title at all.</p> new file mode 100644 @@ -0,0 +1,14 @@ +--- +source: tests/constructs.rs +expression: "render_fixture(\"images.org\")" +--- +<h1 id="bare-image">Bare image</h1> +<p><img src="diagram.png" alt=""></p> +<h1 id="captioned-figure">Captioned figure</h1> +<figure><img src="pipeline.svg" alt="The pipeline, end to end" width="640" class="diagram"><figcaption>The pipeline, end to end</figcaption></figure> +<h1 id="caption-with-markup">Caption with markup</h1> +<figure><img src="chart.png" alt="A stylised chart"><figcaption>A <em>stylised</em> chart</figcaption></figure> +<h1 id="quoted-attribute-values">Quoted attribute values</h1> +<figure><img src="cat.jpg" alt="a cat, sitting" loading="lazy"></figure> +<h1 id="image-with-a-description-is-a-link">Image with a description is a link</h1> +<p><a href="diagram.png">the diagram</a></p> new file mode 100644 @@ -0,0 +1,466 @@ +--- +source: tests/constructs.rs +expression: "parse_fixture(\"lists.org\").root" +--- +{ + "heading": null, + "content": [ + { + "Keyword": { + "key": "TITLE", + "value": "Lists" + } + } + ], + "children": [ + { + "heading": { + "level": 1, + "todo": null, + "priority": null, + "title": [ + { + "Text": "Nesting" + } + ], + "tags": [], + "properties": { + "entries": [] + }, + "id": null, + "custom_id": null + }, + "content": [ + { + "List": { + "kind": "Unordered", + "items": [ + { + "bullet": "Dash", + "checkbox": null, + "term": null, + "content": [ + { + "Paragraph": [ + { + "Text": "outer item" + } + ] + }, + { + "List": { + "kind": "Unordered", + "items": [ + { + "bullet": "Dash", + "checkbox": null, + "term": null, + "content": [ + { + "Paragraph": [ + { + "Text": "inner item" + } + ] + }, + { + "List": { + "kind": "Unordered", + "items": [ + { + "bullet": "Dash", + "checkbox": null, + "term": null, + "content": [ + { + "Paragraph": [ + { + "Text": "deepest item" + } + ] + } + ] + } + ] + } + } + ] + }, + { + "bullet": "Dash", + "checkbox": null, + "term": null, + "content": [ + { + "Paragraph": [ + { + "Text": "second inner" + } + ] + } + ] + } + ] + } + } + ] + }, + { + "bullet": "Dash", + "checkbox": null, + "term": null, + "content": [ + { + "Paragraph": [ + { + "Text": "second outer" + } + ] + } + ] + } + ] + } + } + ], + "children": [] + }, + { + "heading": { + "level": 1, + "todo": null, + "priority": null, + "title": [ + { + "Text": "Ordered" + } + ], + "tags": [], + "properties": { + "entries": [] + }, + "id": null, + "custom_id": null + }, + "content": [ + { + "List": { + "kind": "Ordered", + "items": [ + { + "bullet": { + "Ordered": 1 + }, + "checkbox": null, + "term": null, + "content": [ + { + "Paragraph": [ + { + "Text": "first" + } + ] + } + ] + }, + { + "bullet": { + "Ordered": 2 + }, + "checkbox": null, + "term": null, + "content": [ + { + "Paragraph": [ + { + "Text": "second" + } + ] + }, + { + "List": { + "kind": "Ordered", + "items": [ + { + "bullet": { + "Ordered": 1 + }, + "checkbox": null, + "term": null, + "content": [ + { + "Paragraph": [ + { + "Text": "second point one" + } + ] + } + ] + }, + { + "bullet": { + "Ordered": 2 + }, + "checkbox": null, + "term": null, + "content": [ + { + "Paragraph": [ + { + "Text": "second point two" + } + ] + } + ] + } + ] + } + } + ] + }, + { + "bullet": { + "Ordered": 3 + }, + "checkbox": null, + "term": null, + "content": [ + { + "Paragraph": [ + { + "Text": "third" + } + ] + } + ] + } + ] + } + } + ], + "children": [] + }, + { + "heading": { + "level": 1, + "todo": null, + "priority": null, + "title": [ + { + "Text": "Checkboxes" + } + ], + "tags": [], + "properties": { + "entries": [] + }, + "id": null, + "custom_id": null + }, + "content": [ + { + "List": { + "kind": "Unordered", + "items": [ + { + "bullet": "Dash", + "checkbox": "Off", + "term": null, + "content": [ + { + "Paragraph": [ + { + "Text": "not done" + } + ] + } + ] + }, + { + "bullet": "Dash", + "checkbox": "On", + "term": null, + "content": [ + { + "Paragraph": [ + { + "Text": "done" + } + ] + } + ] + }, + { + "bullet": "Dash", + "checkbox": "Trans", + "term": null, + "content": [ + { + "Paragraph": [ + { + "Text": "partially done" + } + ] + } + ] + } + ] + } + } + ], + "children": [] + }, + { + "heading": { + "level": 1, + "todo": null, + "priority": null, + "title": [ + { + "Text": "Description" + } + ], + "tags": [], + "properties": { + "entries": [] + }, + "id": null, + "custom_id": null + }, + "content": [ + { + "List": { + "kind": "Description", + "items": [ + { + "bullet": "Dash", + "checkbox": null, + "term": [ + { + "Text": "term one" + } + ], + "content": [ + { + "Paragraph": [ + { + "Text": "the first definition" + } + ] + } + ] + }, + { + "bullet": "Dash", + "checkbox": null, + "term": [ + { + "Text": "term two" + } + ], + "content": [ + { + "Paragraph": [ + { + "Text": "the second definition, which is soft-wrapped across two lines" + } + ] + } + ] + }, + { + "bullet": "Dash", + "checkbox": null, + "term": [ + { + "Italic": [ + { + "Text": "marked up" + } + ] + }, + { + "Text": " term" + } + ], + "content": [ + { + "Paragraph": [ + { + "Text": "definitions hold inline markup" + } + ] + } + ] + } + ] + } + } + ], + "children": [] + }, + { + "heading": { + "level": 1, + "todo": null, + "priority": null, + "title": [ + { + "Text": "Multi-paragraph items" + } + ], + "tags": [], + "properties": { + "entries": [] + }, + "id": null, + "custom_id": null + }, + "content": [ + { + "List": { + "kind": "Unordered", + "items": [ + { + "bullet": "Dash", + "checkbox": null, + "term": null, + "content": [ + { + "Paragraph": [ + { + "Text": "an item whose body has two paragraphs" + } + ] + }, + { + "Paragraph": [ + { + "Text": "the second paragraph, indented under the bullet" + } + ] + } + ] + }, + { + "bullet": "Dash", + "checkbox": null, + "term": null, + "content": [ + { + "Paragraph": [ + { + "Text": "a plain sibling" + } + ] + } + ] + } + ] + } + } + ], + "children": [] + } + ] +} new file mode 100644 @@ -0,0 +1,48 @@ +--- +source: tests/constructs.rs +expression: "render_fixture(\"lists.org\")" +--- +<h1 id="nesting">Nesting</h1> +<ul> +<li>outer item<ul> +<li>inner item<ul> +<li>deepest item</li> +</ul> +</li> +<li>second inner</li> +</ul> +</li> +<li>second outer</li> +</ul> +<h1 id="ordered">Ordered</h1> +<ol> +<li>first</li> +<li>second<ol> +<li>second point one</li> +<li>second point two</li> +</ol> +</li> +<li>third</li> +</ol> +<h1 id="checkboxes">Checkboxes</h1> +<ul> +<li><input type="checkbox" disabled> not done</li> +<li><input type="checkbox" disabled checked> done</li> +<li><input type="checkbox" disabled> partially done</li> +</ul> +<h1 id="description">Description</h1> +<dl> +<dt>term one</dt> +<dd>the first definition</dd> +<dt>term two</dt> +<dd>the second definition, which is soft-wrapped across two lines</dd> +<dt><em>marked up</em> term</dt> +<dd>definitions hold inline markup</dd> +</dl> +<h1 id="multi-paragraph-items">Multi-paragraph items</h1> +<ul> +<li><p>an item whose body has two paragraphs</p> +<p>the second paragraph, indented under the bullet</p> +</li> +<li>a plain sibling</li> +</ul> new file mode 100644 @@ -0,0 +1,28 @@ +--- +source: tests/constructs.rs +expression: "render_fixture(\"outofscope.org\")" +--- +<p>Every construct here is on the README's explicit OUT list. The contract is not that we handle them — it is that they degrade predictably and never crash the build.</p> +<h1 id="babel">Babel</h1> +<pre><code class="language-sh highlight"><span class="source shell bash"><span class="meta function-call shell"><span class="support function echo shell">echo</span></span><span class="meta function-call arguments shell"> <span class="string quoted double shell"><span class="punctuation definition string begin shell">"</span>the block renders; :results is never executed<span class="punctuation definition string end shell">"</span></span></span></span></code></pre> +<h1 id="table-formulas">Table formulas</h1> +<table> +<thead> +<tr><th>item</th><th>cost</th></tr> +</thead> +<tbody> +<tr><td>a</td><td>1</td></tr> +<tr><td>b</td><td>2</td></tr> +</tbody> +</table> +<h1 id="latex">LaTeX</h1> +<p>Inline math $x^2 + y^2$ and a display block:</p> +<p>\begin{equation} E = mc^2 \end{equation}</p> +<h1 id="macros-and-radio-targets">Macros and radio targets</h1> +<p>A macro call {{{author}}} and a <<<radio target>>> stay literal.</p> +<h1 id="drawers">Drawers</h1> +<h1 id="verse">Verse</h1> +<pre>An unmodelled block type +keeps its content verbatim.</pre> +<h1 id="entities">Entities</h1> +<p>The full entity set is out of scope, so \alpha stays literal.</p> new file mode 100644 @@ -0,0 +1,209 @@ +--- +source: tests/constructs.rs +expression: "parse_fixture(\"timestamps.org\").root" +--- +{ + "heading": null, + "content": [ + { + "Keyword": { + "key": "TITLE", + "value": "Timestamps" + } + } + ], + "children": [ + { + "heading": { + "level": 1, + "todo": null, + "priority": null, + "title": [ + { + "Text": "Single" + } + ], + "tags": [], + "properties": { + "entries": [] + }, + "id": null, + "custom_id": null + }, + "content": [ + { + "Paragraph": [ + { + "Text": "An active date " + }, + { + "Timestamp": { + "active": true, + "start": "2024-01-15T00:00:00", + "end": null, + "has_time": false + } + }, + { + "Text": " and an inactive one " + }, + { + "Timestamp": { + "active": false, + "start": "2024-01-15T00:00:00", + "end": null, + "has_time": false + } + }, + { + "Text": "." + } + ] + }, + { + "Paragraph": [ + { + "Text": "With a time: " + }, + { + "Timestamp": { + "active": true, + "start": "2024-01-15T10:30:00", + "end": null, + "has_time": true + } + }, + { + "Text": "." + } + ] + } + ], + "children": [] + }, + { + "heading": { + "level": 1, + "todo": null, + "priority": null, + "title": [ + { + "Text": "Ranges" + } + ], + "tags": [], + "properties": { + "entries": [] + }, + "id": null, + "custom_id": null + }, + "content": [ + { + "Paragraph": [ + { + "Text": "A same-day time range " + }, + { + "Timestamp": { + "active": true, + "start": "2024-01-15T10:00:00", + "end": "2024-01-15T11:45:00", + "has_time": true + } + }, + { + "Text": "." + } + ] + }, + { + "Paragraph": [ + { + "Text": "A multi-day range " + }, + { + "Timestamp": { + "active": true, + "start": "2024-01-15T00:00:00", + "end": "2024-01-20T00:00:00", + "has_time": false + } + }, + { + "Text": "." + } + ] + } + ], + "children": [] + }, + { + "heading": { + "level": 1, + "todo": null, + "priority": null, + "title": [ + { + "Text": "Ignored decorations" + } + ], + "tags": [], + "properties": { + "entries": [] + }, + "id": null, + "custom_id": null + }, + "content": [ + { + "Paragraph": [ + { + "Text": "A repeater is dropped: " + }, + { + "Timestamp": { + "active": true, + "start": "2024-01-15T00:00:00", + "end": null, + "has_time": false + } + }, + { + "Text": "." + } + ] + } + ], + "children": [] + }, + { + "heading": { + "level": 1, + "todo": null, + "priority": null, + "title": [ + { + "Text": "Not timestamps" + } + ], + "tags": [], + "properties": { + "entries": [] + }, + "id": null, + "custom_id": null + }, + "content": [ + { + "Paragraph": [ + { + "Text": "Comparisons like 3 < 4 and [not a stamp] stay literal text." + } + ] + } + ], + "children": [] + } + ] +} new file mode 100644 @@ -0,0 +1,14 @@ +--- +source: tests/constructs.rs +expression: "render_fixture(\"timestamps.org\")" +--- +<h1 id="single">Single</h1> +<p>An active date <time class="timestamp" datetime="2024-01-15">2024-01-15</time> and an inactive one <time class="timestamp inactive" datetime="2024-01-15">2024-01-15</time>.</p> +<p>With a time: <time class="timestamp" datetime="2024-01-15T10:30">2024-01-15 10:30</time>.</p> +<h1 id="ranges">Ranges</h1> +<p>A same-day time range <time class="timestamp" datetime="2024-01-15T10:00">2024-01-15 10:00</time>–<time class="timestamp" datetime="2024-01-15T11:45">11:45</time>.</p> +<p>A multi-day range <time class="timestamp" datetime="2024-01-15">2024-01-15</time>–<time class="timestamp" datetime="2024-01-20">2024-01-20</time>.</p> +<h1 id="ignored-decorations">Ignored decorations</h1> +<p>A repeater is dropped: <time class="timestamp" datetime="2024-01-15">2024-01-15</time>.</p> +<h1 id="not-timestamps">Not timestamps</h1> +<p>Comparisons like 3 < 4 and [not a stamp] stay literal text.</p> @@ -5,6 +5,12 @@ expression: "parse_fixture(\"core.org\").root" { "heading": null, "content": [ + { + "Keyword": { + "key": "TITLE", + "value": "Core Constructs" + } + }, { "Paragraph": [ { @@ -14,6 +14,6 @@ expression: "render_fixture(\"core.org\")" </ul> <h1 id="links-and-code">Links and code</h1> <p>An external <a href="https://example.org">site</a> and a bare <a href="https://bare.example">https://bare.example</a>.</p> -<pre><code class="language-rust">fn main() { - println!("hello"); -}</code></pre> +<pre><code class="language-rust highlight"><span class="source rust"><span class="meta function rust"><span class="meta function rust"><span class="storage type function rust">fn</span> </span><span class="entity name function rust">main</span></span><span class="meta function rust"><span class="meta function parameters rust"><span class="punctuation section parameters begin rust">(</span></span><span class="meta function rust"><span class="meta function parameters rust"><span class="punctuation section parameters end rust">)</span></span></span></span><span class="meta function rust"> </span><span class="meta function rust"><span class="meta block rust"><span class="punctuation section block begin rust">{</span> + <span class="support macro rust">println!</span><span class="meta group rust"><span class="punctuation section group begin rust">(</span></span><span class="meta group rust"><span class="string quoted double rust"><span class="punctuation definition string begin rust">"</span>hello<span class="punctuation definition string end rust">"</span></span></span><span class="meta group rust"><span class="punctuation section group end rust">)</span></span><span class="punctuation terminator rust">;</span> +</span><span class="meta block rust"><span class="punctuation section block end rust">}</span></span></span></span></code></pre> @@ -5,6 +5,24 @@ expression: "parse_fixture(\"minimal.org\").root" { "heading": null, "content": [ + { + "Keyword": { + "key": "TITLE", + "value": "Minimal Fixture" + } + }, + { + "Keyword": { + "key": "DATE", + "value": "2026-08-08" + } + }, + { + "Keyword": { + "key": "AUTHOR", + "value": "Owner" + } + }, { "Paragraph": [ { @@ -7,6 +7,7 @@ expression: "page(&pages, \"guide.org\").html" <head> <meta charset="utf-8"> <title>Guide</title> +<link rel="stylesheet" href="syntax.css"> </head> <body> <nav> @@ -7,6 +7,7 @@ expression: "page(&pages, \"index.org\").html" <head> <meta charset="utf-8"> <title>Home</title> +<link rel="stylesheet" href="syntax.css"> </head> <body> <nav>