Commit `0001ef1846`

0001ef1846a43f8b4cb5e92711a7695e29a6c3d7

parent: 903ac6c8d6

Verified · cmc

cmc <hello@cleberg.net> · 2026-08-11 06:48 UTC

v0.15: bundled TOML and Org syntaxes, and org's comma escape

The documentation site written last commit hit both gaps on its first page: syntect
bundles no TOML, and every org-ssg.toml example is TOML; syntect bundles no Org, and a
tool for org users gets written about in org. Both definitions are now written as
.sublime-syntax files under syntaxes/ and compiled into the binary with include_str!, so
they work with no setup like the rest of the zero-config path. 23 code blocks across the
docs went from uncoloured to highlighted.

[highlight] syntaxes_dir loads further .sublime-syntax files from a directory, so the
next missing language is a file rather than a release. A definition that fails to parse
is reported and skipped — one bad file should not stop a site building — while a
malformed *bundled* definition panics, because that is a bug in this crate.

Writing the Org syntax surfaced a real parser bug. Org escapes a line inside a block that
would otherwise look like structure with a leading comma — `,* heading`, `,#+KEYWORD:` —
and the exporter removes exactly one. org-ssg kept it, so documentation *about* org
displayed the escape characters its author had to type, to precisely the audience most
likely to notice. Emacs was consulted and strips them; now so do we.

Checking that against the oracle answered a second question worth recording: an
*unescaped* `*` at column zero inside a block ends the block in Emacs too. Our behaviour
already matched, and my test fixture was invalid org rather than the parser being wrong.
That is why the escape exists at all, and it is now documented.

CACHE_FORMAT_VERSION bumped to 5, because the comma fix changes output for existing
sources and a cache written by the previous binary would otherwise serve the old text.
Noticed only because a rebuild reported "0 rendered" when the output should have changed
— worth remembering that the render key covers inputs, not the binary.

Layout: unified · split

Cargo.lock +1 −1

		@@ -675,7 +675,7 @@ dependencies = [
675	675
676	676	[[package]]
677	677	name = "org-ssg"
678		version = "0.14.0"
	678	version = "0.15.0"
679	679	dependencies = [
680	680	"anyhow",
681	681	"blake3",

Cargo.toml +1 −1

		@@ -1,6 +1,6 @@
1	1	[package]
2	2	name = "org-ssg"
3		version = "0.14.0"
	3	version = "0.15.0"
4	4	edition = "2021"
5	5	description = "Org-mode static site generator that renders the org element tree straight to HTML"
6	6	license = "MIT"

README.md +2 −1

		@@ -364,6 +364,7 @@ all-of-org. Phase 0 checked this line against a real 179-file corpus and found i
364	364	\| 14 \| Authoring: excerpts, word count, reading time, `truncate`, and draft pages \| done \|
365	365	\| 15 \| Table of contents, section numbers, and org's `#+OPTIONS:` per-file switches \| done \|
366	366	\| 16 \| `serve`: development server with long-poll live reload, loopback-bound \| done \|
	367	\| 17 \| Bundled TOML and Org syntaxes, a user syntax directory, and org's comma escape \| done \|
367	368
368	369	### v0.2 in / out
369	370
		@@ -678,7 +679,7 @@ development server), `toml` (config), `chrono`, `camino`, `walkdir`, `clap`, `an
678	679
679	680	```
680	681	cargo build
681		cargo test # 152 tests
	682	cargo test # 156 tests
682	683	cargo run -- init my-site # scaffold a new site
683	684	cargo run -- build fixtures/minimal.org -o minimal.html # single file
684	685	cargo run -- build fixtures/site -o _site # whole site (incremental)

docs/guide/02-configuration.org +10 −3

		@@ -29,6 +29,7 @@ expose_page_list = false
29	29
30	30	[highlight]
31	31	theme = "InspiredGitHub"
	32	syntaxes_dir = "syntaxes"
32	33
33	34	[build]
34	35	drafts = false
		@@ -119,9 +120,10 @@ adding a post a one-page rebuild.
119	120
120	121	* [highlight]
121	122
122		\| Key \| Default \|
123		\|-----+---------\|
124		\| =theme= \| ="InspiredGitHub"= \|
	123	\| Key \| Default \| Meaning \|
	124	\|-----+---------+---------\|
	125	\| =theme= \| ="InspiredGitHub"= \| A syntect theme name. \|
	126	\| =syntaxes_dir= \| ="syntaxes"= \| Extra =.sublime-syntax= files. \|
125	127
126	128	Any theme syntect ships: =InspiredGitHub=, =Solarized (dark)=, =Solarized (light)=,
127	129	=base16-ocean.dark=, =base16-ocean.light=, =base16-eighties.dark=, =base16-mocha.dark=.
		@@ -130,6 +132,11 @@ An unknown name is an error listing the valid ones.
130	132	Highlighting emits CSS classes, never inline styles, so themes live in a stylesheet.
131	133	Each build writes =syntax.css= into the output and every page links it.
132	134
	135	org-ssg bundles TOML and Org on top of syntect's built-in languages. Anything else
	136	missing is a file away: put a =.sublime-syntax= definition in =syntaxes_dir= and it is
	137	loaded. A definition that fails to parse is reported and skipped, because one bad file
	138	should not stop a site from building.
	139
133	140	* [build]
134	141
135	142	\| Key \| Default \| Meaning \|

docs/guide/05-org-support.org +18 −3

		@@ -68,9 +68,24 @@ Recognised, among others: =bash= / =sh=, =c=, =c++=, =css=, =clojure=, =diff=, =
68	68	=makefile=, =markdown=, =matlab=, =objective-c=, =ocaml=, =perl=, =php=, =python=, =r=,
69	69	=ruby=, =rust=, =scala=, =sql=, =tcl=, =xml=, =yaml=.
70	70
71		Not bundled, and worth knowing before you write a page full of them: TOML, INI,
72		Org and Emacs Lisp. The pages of this documentation are a live example — its
73		=#+BEGIN_SRC toml= blocks are readable but uncoloured.
	71	org-ssg adds two syntect does not ship: TOML and Org. Both are what this project's
	72	own documentation needed on its first page — every config example is TOML, and a tool for
	73	org users gets written about in org — so they are compiled into the binary and work with
	74	no setup.
	75
	76	Still missing, and worth knowing before you write a page full of them: INI and *Emacs
	77	Lisp*. For those, drop a =.sublime-syntax= file into the directory named by
	78	=[highlight] syntaxes_dir= (default =syntaxes/=) and it is picked up. A file that fails
	79	to parse is reported and skipped rather than failing the build.
	80
	81	*** The comma escape
	82
	83	A line inside a block that would otherwise look like document structure is written with a
	84	leading comma — =,* heading=, =,#+KEYWORD:= — and org-ssg removes exactly one comma on
	85	output, as Emacs does. Every org example in this documentation relies on it.
	86
	87	The escape is not optional politeness: an unescaped == at column zero ends the block*,
	88	in Emacs as much as here. If a code block seems to stop early, that is why.
74	89
75	90	** Tables and footnotes
76	91

src/config.rs +9

		@@ -259,6 +259,12 @@ impl Default for Templates {
259	259	#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
260	260	#[serde(default, deny_unknown_fields)]
261	261	pub struct Highlight {
	262	/// Directory of extra `.sublime-syntax` files, relative to the source root.
	263	///
	264	/// syntect bundles a long list of languages and this crate adds TOML and Org, but a
	265	/// missing language should not need a new release — drop a definition here and it is
	266	/// picked up. A file that fails to parse is reported and skipped.
	267	pub syntaxes_dir: Utf8PathBuf,
262	268	/// A syntect built-in theme name — `InspiredGitHub`, `Solarized (dark)`,
263	269	/// `base16-ocean.dark`, `base16-eighties.dark`, `base16-mocha.dark`,
264	270	/// `base16-ocean.light`. Highlighting emits CSS classes, and this theme is what the
		@@ -269,6 +275,7 @@ pub struct Highlight {
269	275	impl Default for Highlight {
270	276	fn default() -> Self {
271	277	Highlight {
	278	syntaxes_dir: Utf8PathBuf::from("syntaxes"),
272	279	theme: "InspiredGitHub".to_string(),
273	280	}
274	281	}
		@@ -428,6 +435,8 @@ expose_page_list = false
428	435	# A syntect theme name: InspiredGitHub, Solarized (dark), base16-ocean.dark,
429	436	# base16-eighties.dark, base16-mocha.dark, base16-ocean.light.
430	437	theme = "InspiredGitHub"
	438	# Extra .sublime-syntax files for languages neither syntect nor org-ssg bundles.
	439	syntaxes_dir = "syntaxes"
431	440
432	441	[build]
433	442	# Include pages marked `#+DRAFT:`. Off by default — the point of marking a draft is that

src/incremental.rs +1 −1

		@@ -29,7 +29,7 @@ use crate::util::output_url;
29	29	/// Bump whenever the `Document` type, hashing scheme, or resolution rules change.
30	30	/// On mismatch: discard cache, full rebuild (spec §4.5). The blake3 crate's major
31	31	/// version is folded in as the "hash-algo version" so a hash upgrade also busts.
32		pub const CACHE_FORMAT_VERSION: u32 = 4;
	32	pub const CACHE_FORMAT_VERSION: u32 = 5;
33	33
34	34	/// blake3 hex identity for a content/config/template/render-key hash class (spec §4.1).
35	35	pub type Hash = ContentHash;

src/parser.rs +21 −2

		@@ -525,12 +525,13 @@ fn parse_block(
525	525	base: usize,
526	526	diags: &mut Vec<Diagnostic>,
527	527	) -> (Element, usize) {
528		let mut inner: Vec<&str> = Vec::new();
	528	let mut inner: Vec<String> = Vec::new();
529	529	let mut j = start + 1;
530	530	while j < lines.len() && !is_block_end_of(lines[j], kind) {
531		inner.push(lines[j]);
	531	inner.push(unescape_block_line(lines[j]));
532	532	j += 1;
533	533	}
	534	let inner: Vec<&str> = inner.iter().map(String::as_str).collect();
534	535	if j >= lines.len() {
535	536	// Everything to the end of input was swallowed by the block. This is the single
536	537	// most destructive malformation in org: one missing line silently deletes the
		@@ -569,6 +570,24 @@ fn parse_block(
569	570	(element, next)
570	571	}
571	572
	573	/// Undo org's comma escape on one line of block content.
	574	///
	575	/// A line inside a block that would otherwise look like document structure is written
	576	/// with a leading comma — `,* heading`, `,#+KEYWORD:` — and the exporter removes exactly
	577	/// one comma. Without this, documentation about org shows the escape characters its
	578	/// author had to type, which is precisely the audience most likely to notice.
	579	fn unescape_block_line(line: &str) -> String {
	580	let trimmed = line.trim_start();
	581	let Some(rest) = trimmed.strip_prefix(',') else {
	582	return line.to_string();
	583	};
	584	if !(rest.starts_with('*') \|\| rest.starts_with("#+") \|\| rest.starts_with(',')) {
	585	return line.to_string();
	586	}
	587	let indent = &line[..line.len() - trimmed.len()];
	588	format!("{indent}{rest}")
	589	}
	590
572	591	/// `:NAME:` … `:END:` at block level. A PROPERTIES drawer directly under a heading is
573	592	/// consumed by [`parse_section_body`]; anything reaching here is a generic drawer,
574	593	/// which the renderer drops (README §OUT).

src/render.rs +75 −3

		@@ -42,11 +42,60 @@ pub trait Highlighter {
42	42	/// two must agree or the CSS will not match the markup.
43	43	const CLASS_STYLE: ClassStyle = ClassStyle::Spaced;
44	44
45		/// Syntect's default syntax definitions, loaded once per process (loading is far more
46		/// expensive than highlighting, and a site build highlights many blocks).
	45	/// Syntax definitions syntect does not bundle, compiled into the binary.
	46	///
	47	/// Both are gaps this project hits on its own first page: every `org-ssg.toml` example is
	48	/// TOML, and a tool for org users is going to be written about in org. Embedding them
	49	/// rather than shipping files means they work with no setup, which is the same promise
	50	/// the rest of the zero-config path makes.
	51	const BUNDLED_SYNTAXES: &[(&str, &str)] = &[
	52	("TOML", include_str!("../syntaxes/TOML.sublime-syntax")),
	53	("Org", include_str!("../syntaxes/Org.sublime-syntax")),
	54	];
	55
	56	/// Syntect's default syntax definitions plus [`BUNDLED_SYNTAXES`], loaded once per
	57	/// process (loading is far more expensive than highlighting, and a site build highlights
	58	/// many blocks).
47	59	fn syntax_set() -> &'static SyntaxSet {
48	60	static SET: OnceLock<SyntaxSet> = OnceLock::new();
49		SET.get_or_init(SyntaxSet::load_defaults_newlines)
	61	SET.get_or_init(\|\| build_syntax_set(None))
	62	}
	63
	64	/// Build a syntax set: syntect's defaults, the bundled additions, and optionally a
	65	/// directory of user `.sublime-syntax` files.
	66	///
	67	/// A malformed bundled definition is a bug in this crate and panics. A malformed user
	68	/// definition is reported and skipped, because one bad file in a directory should not
	69	/// stop a site from building.
	70	fn build_syntax_set(user_dir: Option<&camino::Utf8Path>) -> SyntaxSet {
	71	let mut builder = SyntaxSet::load_defaults_newlines().into_builder();
	72	for (name, source) in BUNDLED_SYNTAXES {
	73	let definition =
	74	syntect::parsing::SyntaxDefinition::load_from_str(source, true, Some(name))
	75	.unwrap_or_else(\|e\| panic!("bundled {name} syntax is malformed: {e}"));
	76	builder.add(definition);
	77	}
	78	if let Some(dir) = user_dir.filter(\|d\| d.is_dir()) {
	79	if let Err(e) = builder.add_from_folder(dir, true) {
	80	eprintln!("warning: ignoring syntax definitions in {dir}: {e}");
	81	}
	82	}
	83	builder.build()
	84	}
	85
	86	/// A syntax set including a user directory of `.sublime-syntax` files.
	87	///
	88	/// Cached per directory: a build highlights many blocks, and rebuilding the set for each
	89	/// would cost more than the highlighting.
	90	fn syntax_set_with(user_dir: &camino::Utf8Path) -> &'static SyntaxSet {
	91	use std::collections::HashMap;
	92	use std::sync::Mutex;
	93	static SETS: OnceLock<Mutex<HashMap<camino::Utf8PathBuf, &'static SyntaxSet>>> =
	94	OnceLock::new();
	95	let sets = SETS.get_or_init(\|\| Mutex::new(HashMap::new()));
	96	let mut sets = sets.lock().expect("syntax set cache");
	97	sets.entry(user_dir.to_owned())
	98	.or_insert_with(\|\| Box::leak(Box::new(build_syntax_set(Some(user_dir)))))
50	99	}
51	100
52	101	fn theme_set() -> &'static ThemeSet {
		@@ -81,6 +130,29 @@ impl SyntectHighlighter {
81	130	syntaxes: syntax_set(),
82	131	}
83	132	}
	133
	134	/// A highlighter that also knows the `.sublime-syntax` files in `dir`, for languages
	135	/// neither syntect nor this crate bundles.
	136	pub fn with_syntaxes(dir: Option<&camino::Utf8Path>) -> Self {
	137	match dir {
	138	Some(dir) if dir.is_dir() => SyntectHighlighter {
	139	syntaxes: syntax_set_with(dir),
	140	},
	141	_ => Self::new(),
	142	}
	143	}
	144	}
	145
	146	/// Every language the highlighter recognises, for documentation and error messages.
	147	pub fn available_languages() -> Vec<&'static str> {
	148	let mut names: Vec<&str> = syntax_set()
	149	.syntaxes()
	150	.iter()
	151	.map(\|s\| s.name.as_str())
	152	.collect();
	153	names.sort_unstable();
	154	names.dedup();
	155	names
84	156	}
85	157
86	158	impl Default for SyntectHighlighter {

src/site.rs +1 −1

		@@ -836,7 +836,7 @@ pub fn build_site(src: &Utf8Path, out: &Utf8Path, opts: &BuildOptions) -> Result
836	836	}
837	837	}
838	838
839		let highlighter = SyntectHighlighter::new();
	839	let highlighter = SyntectHighlighter::with_syntaxes(Some(&src.join(&cfg.highlight.syntaxes_dir)));
840	840	let site = site_context(&cfg);
841	841	let listing = page_listing(&cfg, &preps);
842	842	let mut report = SiteReport::default();

syntaxes/Org.sublime-syntax added +119

		@@ -0,0 +1,119 @@
	1	%YAML 1.2
	2	---
	3	# Org mode, for syntect. syntect bundles no Org definition, which is a conspicuous gap in
	4	# a tool whose users write about org — every `#+BEGIN_SRC org` block in the documentation
	5	# needs it.
	6	#
	7	# This highlights org as source text you are reading about, which is a different job
	8	# from parsing it: org-ssg's own parser (src/parser.rs) is what turns org into a
	9	# document. Where the two could disagree, this one stays conservative — a highlighter
	10	# that colours something wrongly is a cosmetic bug, and one that swallows a line is not.
	11	name: Org
	12	file_extensions: [org]
	13	scope: text.org
	14
	15	contexts:
	16	main:
	17	- include: headings
	18	- include: blocks
	19	- include: keywords
	20	- include: comments
	21	- include: lists
	22	- include: tables
	23	- include: inline
	24
	25	headings:
	26	# The whole line is the heading; TODO keyword, priority and tags are picked out
	27	# within it.
	28	- match: '^(\*+)\s'
	29	captures:
	30	1: punctuation.definition.heading.org
	31	push:
	32	- meta_scope: markup.heading.org
	33	- match: $\n?
	34	pop: true
	35	- match: '\b(TODO\|NEXT\|WAITING\|STARTED\|DONE\|CANCELLED\|CANCELED)\b'
	36	scope: keyword.other.todo.org
	37	- match: '\[#[A-Z]\]'
	38	scope: constant.other.priority.org
	39	- match: '(:[A-Za-z0-9_@#%:]+:)\s*$'
	40	scope: entity.name.tag.org
	41	- include: inline
	42
	43	blocks:
	44	# A source block delegates nothing: the inner language is not embedded, because the
	45	# point here is reading org, not rendering the code inside it.
	46	- match: '^\s(#\+(?i:BEGIN_SRC\|BEGIN_EXAMPLE\|BEGIN_QUOTE\|BEGIN_CENTER\|BEGIN_EXPORT\|BEGIN_VERSE\|BEGIN_COMMENT))(.)$'
	47	captures:
	48	1: keyword.control.block.begin.org
	49	2: variable.parameter.org
	50	push:
	51	- meta_scope: markup.raw.block.org
	52	- match: '^\s(#\+(?i:END_\w+))\s$'
	53	captures:
	54	1: keyword.control.block.end.org
	55	pop: true
	56
	57	keywords:
	58	# #+TITLE:, #+DATE:, #+SLUG: and friends — the metadata at the top of a file.
	59	- match: '^\s(#\+[A-Za-z_]+:)(.)$'
	60	captures:
	61	1: keyword.other.keyword.org
	62	2: string.unquoted.org
	63
	64	comments:
	65	# `#` followed by a space. `#+` is a keyword and is matched above.
	66	- match: '^\s#\s.$'
	67	scope: comment.line.org
	68
	69	lists:
	70	- match: '^\s([-+]\|\d+[.)])\s'
	71	scope: punctuation.definition.list.org
	72	- match: '\[[ xX-]\]'
	73	scope: constant.language.checkbox.org
	74
	75	tables:
	76	- match: '^\s\\|.$'
	77	scope: markup.other.table.org
	78
	79	inline:
	80	- include: links
	81	- include: timestamps
	82	- match: '\[fn:[^\]]*\]'
	83	scope: markup.other.footnote.org
	84	# Verbatim and code carry no nested markup, so they come first.
	85	- match: '=[^\s=][^=]*='
	86	scope: markup.raw.inline.org
	87	- match: '~[^\s~][^~]*~'
	88	scope: markup.raw.inline.org
	89	- match: '\[^\s][^]\*'
	90	scope: markup.bold.org
	91	- match: '/[^\s/][^/]*/'
	92	scope: markup.italic.org
	93	- match: '_[^\s_][^_]*_'
	94	scope: markup.underline.org
	95	- match: '\+[^\s+][^+]*\+'
	96	scope: markup.strikethrough.org
	97
	98	links:
	99	- match: '(\[\[)([^\]]*)(\])'
	100	captures:
	101	1: punctuation.definition.link.org
	102	2: markup.underline.link.org
	103	3: punctuation.definition.link.org
	104	push:
	105	- match: '(\[)([^\]]*)(\]\])'
	106	captures:
	107	1: punctuation.definition.link.org
	108	2: string.other.link.title.org
	109	3: punctuation.definition.link.org
	110	pop: true
	111	- match: '\]'
	112	scope: punctuation.definition.link.org
	113	pop: true
	114	- match: ''
	115	pop: true
	116
	117	timestamps:
	118	- match: '[<\[]\d{4}-\d{2}-\d{2}[^>\]]*[>\]]'
	119	scope: constant.other.timestamp.org

syntaxes/TOML.sublime-syntax added +105

		@@ -0,0 +1,105 @@
	1	%YAML 1.2
	2	---
	3	# TOML, for syntect. Not one of the definitions syntect bundles, and the first thing a
	4	# config-heavy site needs — every org-ssg.toml example in the documentation is one.
	5	#
	6	# Scope names are the standard TextMate ones, so any syntect theme colours this without
	7	# knowing it exists.
	8	name: TOML
	9	file_extensions: [toml, tml]
	10	scope: source.toml
	11
	12	contexts:
	13	main:
	14	- include: comments
	15	- include: tables
	16	- include: keys
	17	- include: values
	18
	19	comments:
	20	- match: '#'
	21	scope: punctuation.definition.comment.toml
	22	push:
	23	- meta_scope: comment.line.number-sign.toml
	24	- match: $\n?
	25	pop: true
	26
	27	tables:
	28	# [[array.of.tables]] before [table], so the doubled bracket wins.
	29	- match: '^\s(\[\[)([^\]])(\]\])'
	30	captures:
	31	1: punctuation.definition.table.array.toml
	32	2: entity.name.section.toml
	33	3: punctuation.definition.table.array.toml
	34	- match: '^\s(\[)([^\]])(\])'
	35	captures:
	36	1: punctuation.definition.table.toml
	37	2: entity.name.section.toml
	38	3: punctuation.definition.table.toml
	39
	40	keys:
	41	- match: '([A-Za-z0-9_\-]+\|"[^"]"\|''[^'']'')\s*(=)'
	42	captures:
	43	1: variable.other.key.toml
	44	2: keyword.operator.assignment.toml
	45
	46	values:
	47	- include: strings
	48	- match: '\b(true\|false)\b'
	49	scope: constant.language.toml
	50	# Dates and times before numbers, or the year is read as an integer.
	51	- match: '\b\d{4}-\d{2}-\d{2}([Tt ]\d{2}:\d{2}:\d{2}(\.\d+)?([Zz]\|[+\-]\d{2}:\d{2})?)?'
	52	scope: constant.numeric.date.toml
	53	- match: '\b\d{2}:\d{2}:\d{2}(\.\d+)?'
	54	scope: constant.numeric.time.toml
	55	- match: '\b0x[0-9A-Fa-f_]+\b'
	56	scope: constant.numeric.hex.toml
	57	- match: '\b0o[0-7_]+\b'
	58	scope: constant.numeric.oct.toml
	59	- match: '\b0b[01_]+\b'
	60	scope: constant.numeric.bin.toml
	61	- match: '[+\-]?\b\d[\d_]*(\.[\d_]+)?([eE][+\-]?\d+)?\b'
	62	scope: constant.numeric.toml
	63	- match: '\b(inf\|nan)\b'
	64	scope: constant.numeric.toml
	65
	66	strings:
	67	# Multi-line forms first: """ would otherwise match as an empty "" plus a stray ".
	68	- match: '"""'
	69	scope: punctuation.definition.string.begin.toml
	70	push:
	71	- meta_scope: string.quoted.double.block.toml
	72	- match: '"""'
	73	scope: punctuation.definition.string.end.toml
	74	pop: true
	75	- include: escapes
	76	- match: "'''"
	77	scope: punctuation.definition.string.begin.toml
	78	push:
	79	- meta_scope: string.quoted.single.block.toml
	80	- match: "'''"
	81	scope: punctuation.definition.string.end.toml
	82	pop: true
	83	- match: '"'
	84	scope: punctuation.definition.string.begin.toml
	85	push:
	86	- meta_scope: string.quoted.double.toml
	87	- match: '"'
	88	scope: punctuation.definition.string.end.toml
	89	pop: true
	90	- match: $\n?
	91	pop: true
	92	- include: escapes
	93	- match: "'"
	94	scope: punctuation.definition.string.begin.toml
	95	push:
	96	- meta_scope: string.quoted.single.toml
	97	- match: "'"
	98	scope: punctuation.definition.string.end.toml
	99	pop: true
	100	- match: $\n?
	101	pop: true
	102
	103	escapes:
	104	- match: '\\(u[0-9A-Fa-f]{4}\|U[0-9A-Fa-f]{8}\|[btnfr"\\/]\|\s*\n)'
	105	scope: constant.character.escape.toml

tests/constructs.rs +111

		@@ -391,3 +391,114 @@ fn well_formed_fixtures_produce_no_diagnostics() {
391	391	);
392	392	}
393	393	}
	394
	395	// ---------------------------------------------------------------------------
	396	// Bundled syntax definitions, and org's comma escape
	397	// ---------------------------------------------------------------------------
	398
	399	/// syntect bundles neither TOML nor Org. Both are gaps this project hits on its own
	400	/// first documentation page: every config example is TOML, and a tool for org users gets
	401	/// written about in org.
	402	#[test]
	403	fn toml_and_org_blocks_are_highlighted() {
	404	for (lang, code, expect_scope) in [
	405	(
	406	"toml",
	407	"# comment\n[site]\ntitle = \"x\"\nport = 3000\nok = true\n",
	408	"entity name section toml",
	409	),
	410	(
	411	// The heading is comma-escaped, which org requires inside a block: an
	412	// unescaped `*` at column 0 ends the block in Emacs too, verified against it.
	413	"org",
	414	",#+TITLE: A page\n\n,* TODO [#A] Heading :tag:\n\nSome bold text.\n",
	415	"markup heading org",
	416	),
	417	] {
	418	let source = format!("#+BEGIN_SRC {lang}\n{code}#+END_SRC\n");
	419	let document = parse(Utf8PathBuf::from("t.org").as_path(), &source).expect("parse");
	420	let Html(html) = render(&ResolvedDoc { document }, &SyntectHighlighter::new());
	421
	422	assert!(
	423	html.contains(&format!("class=\"language-{lang} highlight\"")),
	424	"{lang} should be highlighted, not fall back to plain code:\n{html}"
	425	);
	426	assert!(
	427	html.contains(expect_scope),
	428	"{lang} should produce the scope {expect_scope:?}:\n{html}"
	429	);
	430	}
	431	}
	432
	433	/// TOML's lexical corners: a table array is not a table, a date is not an integer, and a
	434	/// comment is not a table header.
	435	#[test]
	436	fn the_toml_syntax_distinguishes_its_shapes() {
	437	let code = "#+BEGIN_SRC toml\n# note\n[[collections]]\nwhen = 2026-08-11\nn = 12\ns = \"q\"\nb = false\n#+END_SRC\n";
	438	let document = parse(Utf8PathBuf::from("t.org").as_path(), code).expect("parse");
	439	let Html(html) = render(&ResolvedDoc { document }, &SyntectHighlighter::new());
	440
	441	for scope in [
	442	"comment line number-sign toml",
	443	"entity name section toml",
	444	"constant numeric date toml",
	445	"string quoted double toml",
	446	"constant language toml",
	447	] {
	448	assert!(html.contains(scope), "expected scope {scope:?}:\n{html}");
	449	}
	450	}
	451
	452	/// Org escapes a line inside a block that would look like structure by prefixing a
	453	/// comma, and the exporter removes it. Without this, documentation about org shows the
	454	/// escape characters its author had to type — to exactly the audience most likely to
	455	/// notice. Verified against Emacs, which strips them.
	456	#[test]
	457	fn the_comma_escape_is_removed_from_block_content() {
	458	let source = concat!(
	459	"#+BEGIN_SRC org\n",
	460	",#+TITLE: A page\n",
	461	",* A heading\n",
	462	",,* not a heading, one comma removed\n",
	463	"plain line\n",
	464	"#+END_SRC\n",
	465	);
	466	let document = parse(Utf8PathBuf::from("t.org").as_path(), source).expect("parse");
	467	let Html(html) = render(&ResolvedDoc { document }, &SyntectHighlighter::new());
	468	// Highlighting splits the line across spans, so compare the text, not the markup.
	469	let text = strip_tags(&html);
	470
	471	assert!(text.contains("#+TITLE: A page"), "the comma is gone:\n{html}");
	472	assert!(!text.contains(",#+TITLE:"), "and not merely moved:\n{html}");
	473	assert!(
	474	text.contains(",* not a heading"),
	475	"a doubled comma loses exactly one:\n{html}"
	476	);
	477	assert!(text.contains("plain line"), "other lines are untouched:\n{html}");
	478	}
	479
	480	/// Text content of an HTML fragment, with tags removed and entities decoded.
	481	fn strip_tags(html: &str) -> String {
	482	let mut text = String::new();
	483	let mut rest = html;
	484	while let Some(open) = rest.find('<') {
	485	text.push_str(&rest[..open]);
	486	match rest[open..].find('>') {
	487	Some(close) => rest = &rest[open + close + 1..],
	488	None => break,
	489	}
	490	}
	491	text.push_str(rest);
	492	text.replace("&", "&").replace("<", "<").replace(">", ">")
	493	}
	494
	495	/// A comma that is not an escape is content, and must survive.
	496	#[test]
	497	fn an_ordinary_leading_comma_is_not_stripped() {
	498	let source = "#+BEGIN_SRC text\n, a list continuation\n,not an escape\n#+END_SRC\n";
	499	let document = parse(Utf8PathBuf::from("t.org").as_path(), source).expect("parse");
	500	let Html(html) = render(&ResolvedDoc { document }, &SyntectHighlighter::new());
	501	let text = strip_tags(&html);
	502	assert!(text.contains(", a list continuation"), "{html}");
	503	assert!(text.contains(",not an escape"), "{html}");
	504	}

Commit 0001ef1846

Commit `0001ef1846`