Commit `20a54d84ca`

20a54d84ca28c93ebb442f224df7398c348938dc

parent: ecaccb90aa

Verified · cmc

cmc <hello@cleberg.net> · 2026-08-11 06:06 UTC

v0.14: serve command with live reload

`watch` rebuilt but left you to serve the output and press reload. This closes the loop:
build, watch, serve, and reload the browser when a rebuild lands.

Three decisions shape it:

Loopback by default. A development server serves unreviewed drafts off someone's laptop,
so reaching the local network is something you ask for with --host, never something you
get.

The reload script is injected on the way out, never written to disk. What you deploy is
the built site and it must not carry a dev server's JavaScript — there is a test that the
file on disk stays clean while the served response does not.

Long-polling rather than WebSockets or SSE. The browser asks "anything since generation
N?" and the server holds the request until there is; the generation the page was built
from is baked into the injected script, so a rebuild landing between serving a page and
its first poll is answered immediately rather than leaving the tab stale. A timeout
answers with the current generation, so a client whose counter is ahead of a restarted
server self-corrects instead of hanging forever.

SSE was the first attempt and did not work: tiny_http buffers a response until its body
ends, so a body that never ends never reaches the client — not one byte, not even the
headers. Long-polling uses tiny_http exactly as designed.

URL resolution is the server's security boundary, so it is a pure function with its own
tests rather than something only reachable through a running server. Paths are built
from accepted segments rather than normalized after joining, because it is far easier to
reason about a list that never contained a `..` than about removing one correctly.
Traversal via `..`, percent-encoded `..`, backslashes, absolute paths and embedded NULs
all resolve to nothing, verified both directly and against the running server.

Mutation-testing that guard was worth it twice: replacing `..`-rejection with
`..`-popping did *not* fail the tests, and should not have — a path built from an empty
vec cannot pop below its root, so that variant is equally safe. Treating `..` as an
ordinary segment does fail them, which is the mutation that matters, because the
textual `starts_with(root)` check alone would have let `out/../secret.txt` through.

All 182 incumbent URLs still reproduced.

Layout: unified · split

Cargo.lock +32 −1

 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "d3fb67a6e08acf24fdeccbac2cb6ac4305825bd1f117462e0e6f2f193345ad56"
+[[package]]
+name = "ascii"
+version = "1.1.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "d92bec98840b8f03a5ff5413de5293bfcd8bf96467cf5452609f939ec6f5de16"
 [[package]]
 name = "autocfg"
 version = "1.5.1"
  "windows-link",
 ]
+[[package]]
+name = "chunked_transfer"
+version = "1.5.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "6e4de3bc4ea267985becf712dc6d9eed8b04c953b3fcfb339ebc87acd9804901"
 [[package]]
 name = "clap"
 version = "4.6.6"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "2304e00983f87ffb38b55b444b5e3b60a884b5d30c0fca7d82fe33449bbe55ea"
+[[package]]
+name = "httpdate"
+version = "1.0.3"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "df3b46402a9d5adb4c86a0cf463f42e19994e3ee891101b1841f30a545cb49a9"
 [[package]]
 name = "iana-time-zone"
 version = "0.1.65"
 [[package]]
 name = "org-ssg"
-version = "0.13.0"
+version = "0.14.0"
 dependencies = [
  "anyhow",
  "blake3",
  "serde_json",
  "syntect",
  "thiserror",
+ "tiny_http",
  "toml",
  "walkdir",
 ]
  "time-core",
 ]
+[[package]]
+name = "tiny_http"
+version = "0.12.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "389915df6413a2e74fb181895f933386023c71110878cd0825588928e64cdc82"
+dependencies = [
+ "ascii",
+ "chunked_transfer",
+ "httpdate",
+ "log",
+]
 [[package]]
 name = "toml"
 version = "1.1.4+spec-1.1.0"

Cargo.toml +2 −1

 [package]
 name = "org-ssg"
-version = "0.13.0"
+version = "0.14.0"
 edition = "2021"
 description = "Org-mode static site generator that renders the org element tree straight to HTML"
 license = "MIT"
 rayon = "1.12.0"
 toml = "1.1.4"
 notify = "8.2.0"
+tiny_http = "0.12.0"
 [dev-dependencies]
 insta = { version = "1", features = ["json"] }

README.md +31 −2

 | **13** | **`watch` on OS filesystem events, debounced, with the feedback loop closed** | **done** |
 | **14** | **Authoring: excerpts, word count, reading time, `truncate`, and draft pages** | **done** |
 | **15** | **Table of contents, section numbers, and org's `#+OPTIONS:` per-file switches** | **done** |
+| **16** | **`serve`: development server with long-poll live reload, loopback-bound** | **done** |
 ### v0.2 in / out
 (`SCHEDULED:`/`DEADLINE:`), which render as ordinary paragraphs; and fixed-width `: `
 lines.
+## Serving
+```bash
+cargo run -- serve my-site -o _site        # http://127.0.0.1:3000
+```
+Builds, watches, serves, and reloads the browser when a rebuild lands — the loop `watch`
+leaves half-open.
+- **Loopback by default.** A dev server serves unreviewed drafts off your laptop, so
+  reaching the local network is something you ask for with `--host 0.0.0.0`, never
+  something you get.
+- **The reload script is injected on the way out**, never written to disk. What you
+  deploy is the built site, and it must not carry a dev server's JavaScript.
+- **Long-polling, not WebSockets or SSE.** The browser asks "anything since generation
+  N?" and the server holds the request until there is. Instant like a push, no protocol
+  beyond ordinary HTTP, and no dependency. A streamed response would have been more
+  elegant and does not work: tiny_http buffers a response until its body ends, so a body
+  that never ends never reaches the client.
+- A reload only follows a **successful** rebuild. Reloading onto a stale page because the
+  build just failed tells you nothing; the error is already on your terminal.
+URL resolution is the server's security boundary and is written as a pure function with
+its own tests: `..`, percent-encoded `..`, backslashes, absolute paths and embedded NULs
+all resolve to nothing rather than to somewhere outside the output directory.
 ## Watching
 ```bash
 line-oriented and context-sensitive, not clean CFG). Key crates: `syntect` (syntax
 highlighting, behind a `Highlighter` trait so tree-sitter can be swapped in later),
 `minijinja` (runtime templates), `blake3` (content/cache hashing), `rayon` (parallel
-PARSE/RESOLVE/RENDER), `notify` (filesystem events for `watch`), `toml` (config), `chrono`, `camino`, `walkdir`, `clap`, `anyhow`/`thiserror`.
+PARSE/RESOLVE/RENDER), `notify` (filesystem events for `watch`), `tiny_http` (the `serve`
+development server), `toml` (config), `chrono`, `camino`, `walkdir`, `clap`, `anyhow`/`thiserror`.
 `insta` for snapshot tests, and `emacs --batch` — optional, and only for the oracle.
 ## Build & test
 ```
 cargo build
-cargo test                                                # 142 tests
+cargo test                                                # 152 tests
 cargo run -- init my-site                                 # scaffold a new site
 cargo run -- build fixtures/minimal.org -o minimal.html   # single file
 cargo run -- build fixtures/site -o _site                 # whole site (incremental)
 cargo run -- audit fixtures/site                          # corpus audit (Phase 0)
 cargo run -- build fixtures/site -o _site --no-cache      # force a full rebuild
 cargo run -- watch fixtures/site -o _site                 # rebuild on filesystem events
+cargo run -- serve fixtures/site -o _site                 # ... and serve with live reload
 cargo run -- clean _site                                  # remove output + cache
 ```

src/lib.rs +1

	@@ -16,6 +16,7 @@ pub mod model;
16	pub mod parser;	16	pub mod parser;
17	pub mod render;	17	pub mod render;
18	pub mod resolve;	18	pub mod resolve;
		19	pub mod serve;
19	pub mod site;	20	pub mod site;
20	pub mod template;	21	pub mod template;
21	pub mod util;	22	pub mod util;

src/main.rs +39

	@@ -64,6 +64,27 @@ enum Command {
64	#[arg(long)]	64	#[arg(long)]
65	drafts: bool,	65	drafts: bool,
66	},	66	},
		67	/// Serve the built site locally, rebuilding and reloading the browser on change.
		68	Serve {
		69	/// Source directory to build and watch.
		70	input: Utf8PathBuf,
		71	/// Output directory to serve.
		72	#[arg(short, long)]
		73	output: Utf8PathBuf,
		74	/// Port to listen on.
		75	#[arg(short, long, default_value_t = 3000)]
		76	port: u16,
		77	/// Address to bind. Defaults to loopback; set `0.0.0.0` to expose the server to
		78	/// your network, which also exposes any drafts you are building.
		79	#[arg(long, default_value = "127.0.0.1")]
		80	host: String,
		81	/// Include pages marked `#+DRAFT:`.
		82	#[arg(long)]
		83	drafts: bool,
		84	/// Config file to use, overriding `org-ssg.toml` in the source directory.
		85	#[arg(long, value_name = "FILE")]
		86	config: Option<Utf8PathBuf>,
		87	},
67	/// Remove the build output directory (which holds the cache manifest).	88	/// Remove the build output directory (which holds the cache manifest).
68	Clean {	89	Clean {
69	/// Output directory to remove.	90	/// Output directory to remove.
	@@ -145,6 +166,24 @@ fn main() -> Result<()> {
145	print!("{}", org_ssg::audit::report(&result));	166	print!("{}", org_ssg::audit::report(&result));
146	Ok(())	167	Ok(())
147	}	168	}
		169	Command::Serve {
		170	input,
		171	output,
		172	port,
		173	host,
		174	drafts,
		175	config,
		176	} => org_ssg::serve::run(
		177	&input,
		178	&output,
		179	&BuildOptions {
		180	drafts,
		181	config_path: config,
		182	..Default::default()
		183	},
		184	&host,
		185	port,
		186	),
148	Command::Init { directory } => init(&directory),	187	Command::Init { directory } => init(&directory),
149	Command::Clean { output } => {	188	Command::Clean { output } => {
150	if output.exists() {	189	if output.exists() {

src/serve.rs added +300

@@ -0,0 +1,300 @@
	1	//! `serve`: a development server over the built site, with browser live reload.
	2	//!
	3	//! `watch` rebuilds but leaves you to serve the output and press reload yourself. This
	4	//! closes that loop: build, watch, serve, and push a reload to the browser when a
	5	//! rebuild lands.
	6	//!
	7	//! Three decisions shape it:
	8	//!
	9	//! 1. Loopback by default. A development server binds `127.0.0.1`, not `0.0.0.0`.
	10	//! It serves unreviewed drafts off someone's laptop, and exposing that to the local
	11	//! network should be a thing you ask for (`--host`), never a thing you get.
	12	//! 2. The reload script is injected at serve time, never written to disk. The built
	13	//! site is what you deploy, and it must not carry a dev server's JavaScript.
	14	//! 3. Long-polling, not WebSockets or SSE. The browser asks "has anything changed
	15	//! since generation N?" and the server holds the request open until something has.
	16	//! That is instant like a push, needs no protocol beyond ordinary HTTP, and — unlike
	17	//! a streamed response — completes, which is what makes it work at all: tiny_http
	18	//! buffers a response until its body ends, so a body that never ends never reaches
	19	//! the client. Long-polling was the version of this that worked.
	20
	21	use std::io;
	22	use std::sync::{Arc, Condvar, Mutex};
	23	use std::time::Duration;
	24
	25	use anyhow::Result;
	26	use camino::{Utf8Path, Utf8PathBuf};
	27	use tiny_http::{Header, Request, Response, Server, StatusCode};
	28
	29	use crate::site::{build_site, BuildOptions};
	30
	31	/// Where the browser subscribes for reload events. Namespaced so it cannot collide with
	32	/// a real page.
	33	pub const RELOAD_PATH: &str = "/__org-ssg/reload";
	34
	35	/// How long a poll waits before answering "nothing yet". Long enough that an idle tab is
	36	/// nearly silent, short enough to stay under any proxy or browser idle timeout.
	37	const POLL_TIMEOUT: Duration = Duration::from_secs(25);
	38
	39	/// The script injected into served HTML, carrying the generation the page was built
	40	/// from.
	41	///
	42	/// Baking the generation in is what makes this race-free: if a rebuild lands between the
	43	/// page being served and the first poll going out, the server answers immediately rather
	44	/// than the tab sitting on stale content until the next edit.
	45	fn reload_script(generation: u64) -> String {
	46	format!(
	47	"\n<script>(function p(n){{fetch(\"{RELOAD_PATH}?since=\"+n)\
	48	.then(function(r){{return r.json()}})\
	49	.then(function(g){{g>n?location.reload():p(g)}})\
	50	.catch(function(){{setTimeout(function(){{p(n)}},1000)}})}})({generation})</script>\n"
	51	)
	52	}
	53
	54	/// A build counter that event streams wait on.
	55	#[derive(Default)]
	56	struct BuildSignal {
	57	generation: Mutex<u64>,
	58	changed: Condvar,
	59	}
	60
	61	impl BuildSignal {
	62	fn bump(&self) {
	63	*self.generation.lock().expect("build signal") += 1;
	64	self.changed.notify_all();
	65	}
	66	}
	67
	68	/// Run the development server until interrupted.
	69	pub fn run(
	70	src: &Utf8Path,
	71	out: &Utf8Path,
	72	opts: &BuildOptions,
	73	host: &str,
	74	port: u16,
	75	) -> Result<()> {
	76	if !src.is_dir() {
	77	anyhow::bail!("serve requires a source directory: serve <src-dir> -o <out-dir>");
	78	}
	79	let report = build_site(src, out, opts)?;
	80
	81	let address = format!("{host}:{port}");
	82	let server = Server::http(&address).map_err(\|e\| {
	83	anyhow::anyhow!("cannot listen on {address}: {e}. Is something already using port {port}?")
	84	})?;
	85	let server = Arc::new(server);
	86	let signal = Arc::new(BuildSignal::default());
	87	let root: Utf8PathBuf = out.to_owned();
	88
	89	println!(
	90	"serving {} page(s) from {out} at http://{address}/ — Ctrl-C to stop.",
	91	report.pages.len()
	92	);
	93
	94	// Rebuild in the background; the main thread serves.
	95	{
	96	let (src, out, opts, signal) = (
	97	src.to_owned(),
	98	out.to_owned(),
	99	opts.clone(),
	100	Arc::clone(&signal),
	101	);
	102	std::thread::spawn(move \|\| {
	103	let result = crate::watch::run_with(&src, &out, &opts, \|built\| {
	104	// Reload on a successful rebuild only. Reloading onto a stale page
	105	// because the build just failed tells the author nothing; the error is
	106	// already on their terminal.
	107	if built.is_ok() {
	108	signal.bump();
	109	}
	110	});
	111	if let Err(e) = result {
	112	eprintln!("watch stopped: {e:#}");
	113	}
	114	});
	115	}
	116
	117	// A thread per request. The volume is one developer's browser, and an event stream
	118	// occupies its thread for as long as the tab is open — which a fixed pool would let
	119	// starve everything else.
	120	for request in server.incoming_requests() {
	121	let root = root.clone();
	122	let signal = Arc::clone(&signal);
	123	std::thread::spawn(move \|\| {
	124	if let Err(e) = handle(request, &root, &signal) {
	125	// A browser closing a tab mid-response is routine, not a problem.
	126	if e.kind() != io::ErrorKind::BrokenPipe {
	127	eprintln!("serve: {e}");
	128	}
	129	}
	130	});
	131	}
	132	Ok(())
	133	}
	134
	135	fn handle(request: Request, root: &Utf8Path, signal: &Arc<BuildSignal>) -> io::Result<()> {
	136	let url = request.url().to_string();
	137	if url.split(['?', '#']).next() == Some(RELOAD_PATH) {
	138	let since = since_parameter(&url);
	139	return serve_poll(request, signal, since);
	140	}
	141	match resolve(root, &url) {
	142	Some(path) => serve_file(request, &path, signal),
	143	None => request.respond(
	144	Response::from_string("404 not found")
	145	.with_status_code(StatusCode(404))
	146	.with_header(header("Content-Type", "text/plain; charset=utf-8")),
	147	),
	148	}
	149	}
	150
	151	fn serve_file(request: Request, path: &Utf8Path, signal: &Arc<BuildSignal>) -> io::Result<()> {
	152	let Ok(bytes) = std::fs::read(path) else {
	153	return request.respond(
	154	Response::from_string("404 not found").with_status_code(StatusCode(404)),
	155	);
	156	};
	157	let mime = mime_type(path);
	158	let bytes = if mime.starts_with("text/html") {
	159	let generation = *signal.generation.lock().expect("build signal");
	160	inject_reload_script(&bytes, generation)
	161	} else {
	162	bytes
	163	};
	164	request.respond(
	165	Response::from_data(bytes)
	166	.with_header(header("Content-Type", mime))
	167	// A dev server must never be cached, or an edit appears not to have landed.
	168	.with_header(header("Cache-Control", "no-store")),
	169	)
	170	}
	171
	172	/// Put the reload script just before `</body>`, or at the end if there is none.
	173	pub fn inject_reload_script(bytes: &[u8], generation: u64) -> Vec<u8> {
	174	let Ok(text) = std::str::from_utf8(bytes) else {
	175	return bytes.to_vec();
	176	};
	177	let script = reload_script(generation);
	178	match text.rfind("</body>") {
	179	Some(at) => format!("{}{script}{}", &text[..at], &text[at..]).into_bytes(),
	180	None => format!("{text}{script}").into_bytes(),
	181	}
	182	}
	183
	184	/// Answer a poll: block until the build generation passes `since`, then report it.
	185	///
	186	/// A timeout answers with the current generation, which the client compares itself —
	187	/// so a slow answer is indistinguishable from a fast one and no event can be missed.
	188	fn serve_poll(request: Request, signal: &Arc<BuildSignal>, since: u64) -> io::Result<()> {
	189	let guard = signal.generation.lock().expect("build signal");
	190	let (guard, _) = signal
	191	.changed
	192	.wait_timeout_while(guard, POLL_TIMEOUT, \|generation\| *generation <= since)
	193	.expect("build signal");
	194	let generation = *guard;
	195	drop(guard);
	196
	197	request.respond(
	198	Response::from_string(generation.to_string())
	199	.with_header(header("Content-Type", "application/json"))
	200	.with_header(header("Cache-Control", "no-store")),
	201	)
	202	}
	203
	204	/// The `since=N` parameter of a poll request.
	205	pub fn since_parameter(url: &str) -> u64 {
	206	url.split_once('?')
	207	.map(\|(_, query)\| query)
	208	.into_iter()
	209	.flat_map(\|query\| query.split('&'))
	210	.find_map(\|pair\| pair.strip_prefix("since="))
	211	.and_then(\|value\| value.parse().ok())
	212	.unwrap_or(0)
	213	}
	214
	215	/// Map a request URL onto a file inside `root`, or `None` if it does not name one.
	216	///
	217	/// This is the server's security boundary, so it is a pure function with its own tests.
	218	/// A URL is attacker-controlled input even on a development server: `..` segments,
	219	/// percent-encoded `..`, absolute paths and backslashes all have to resolve to nothing
	220	/// rather than to somewhere outside the output directory.
	221	pub fn resolve(root: &Utf8Path, url: &str) -> Option<Utf8PathBuf> {
	222	let path = url.split(['?', '#']).next().unwrap_or("");
	223	let decoded = percent_decode(path);
	224
	225	// Build the path from scratch out of accepted segments. Normalizing a joined path
	226	// afterwards is the version of this that has bugs: it is far easier to reason about
	227	// a list that never contained a `..` than about removing one correctly.
	228	let mut segments: Vec<&str> = Vec::new();
	229	for segment in decoded.split(['/', '\\']) {
	230	match segment {
	231	"" \| "." => {}
	232	".." => return None,
	233	// A NUL or a path separator that survived decoding is not a filename.
	234	s if s.contains('\0') => return None,
	235	s => segments.push(s),
	236	}
	237	}
	238
	239	let mut candidate = root.to_owned();
	240	for segment in &segments {
	241	candidate.push(segment);
	242	}
	243	// Directories, and the bare root, serve their index.
	244	if decoded.ends_with('/') \|\| segments.is_empty() \|\| candidate.is_dir() {
	245	candidate.push("index.html");
	246	}
	247	// Defence in depth: whatever the segment logic did, the result must be inside root.
	248	if !candidate.starts_with(root) {
	249	return None;
	250	}
	251	candidate.is_file().then_some(candidate)
	252	}
	253
	254	/// Decode `%XX` escapes. `+` is left alone: it means a space in a query string, not in a
	255	/// path, and turning `a+b.html` into `a b.html` would break a real filename.
	256	fn percent_decode(input: &str) -> String {
	257	let bytes = input.as_bytes();
	258	let mut out: Vec<u8> = Vec::with_capacity(bytes.len());
	259	let mut i = 0;
	260	while i < bytes.len() {
	261	if bytes[i] == b'%' && i + 2 < bytes.len() {
	262	let hex = std::str::from_utf8(&bytes[i + 1..i + 3]).ok();
	263	if let Some(byte) = hex.and_then(\|h\| u8::from_str_radix(h, 16).ok()) {
	264	out.push(byte);
	265	i += 3;
	266	continue;
	267	}
	268	}
	269	out.push(bytes[i]);
	270	i += 1;
	271	}
	272	String::from_utf8_lossy(&out).into_owned()
	273	}
	274
	275	fn mime_type(path: &Utf8Path) -> &'static str {
	276	match path.extension().unwrap_or("").to_ascii_lowercase().as_str() {
	277	"html" \| "htm" => "text/html; charset=utf-8",
	278	"css" => "text/css; charset=utf-8",
	279	"js" => "text/javascript; charset=utf-8",
	280	"json" => "application/json",
	281	"xml" \| "rss" \| "atom" => "application/xml; charset=utf-8",
	282	"txt" => "text/plain; charset=utf-8",
	283	"svg" => "image/svg+xml",
	284	"png" => "image/png",
	285	"jpg" \| "jpeg" => "image/jpeg",
	286	"gif" => "image/gif",
	287	"webp" => "image/webp",
	288	"avif" => "image/avif",
	289	"ico" => "image/x-icon",
	290	"woff2" => "font/woff2",
	291	"woff" => "font/woff",
	292	"ttf" => "font/ttf",
	293	"pdf" => "application/pdf",
	294	_ => "application/octet-stream",
	295	}
	296	}
	297
	298	fn header(name: &str, value: &str) -> Header {
	299	Header::from_bytes(name.as_bytes(), value.as_bytes()).expect("static header is valid")
	300	}

src/watch.rs +14 −1

 /// Build once, then rebuild whenever the source changes. Runs until interrupted.
 pub fn run(src: &Utf8Path, out: &Utf8Path, opts: &BuildOptions) -> Result<()> {
+    run_with(src, out, opts, |_| {})
+}
+/// As [`run`], calling `on_rebuild` after each rebuild attempt — which is how `serve`
+/// learns that it has something new to tell the browser.
+pub fn run_with(
+    src: &Utf8Path,
+    out: &Utf8Path,
+    opts: &BuildOptions,
+    mut on_rebuild: impl FnMut(&Result<crate::site::SiteReport>),
+) -> Result<()> {
     if !src.is_dir() {
         anyhow::bail!("watch requires a source directory: watch <src-dir> -o <out-dir>");
     }
         }
         let summary = summarize(&changed);
-        match build_site(src, out, opts) {
+        let result = build_site(src, out, opts);
+        match &result {
             Ok(report) => println!(
                 "{summary}: {} rendered, {} cached",
                 report.rendered.len(),
             // half-saved file, and the next keystroke fixes it.
             Err(e) => eprintln!("{summary}: build failed: {e:#}"),
         }
+        on_rebuild(&result);
     }
 }

tests/serve.rs added +240

@@ -0,0 +1,240 @@
	1	//! `serve`: URL resolution, reload-script injection, and one live server.
	2	//!
	3	//! `resolve` is the server's security boundary. A URL is attacker-controlled input even
	4	//! on a development server — a page someone is previewing can contain a link, an image
	5	//! or a fetch to anything — so it gets tested as the pure function it deliberately is,
	6	//! rather than only through a running server.
	7
	8	use std::io::{Read, Write};
	9	use std::net::TcpStream;
	10	use std::sync::atomic::{AtomicU32, Ordering};
	11	use std::time::{Duration, Instant};
	12
	13	use camino::{Utf8Path, Utf8PathBuf};
	14
	15	use org_ssg::serve::{inject_reload_script, resolve, since_parameter};
	16	use org_ssg::site::BuildOptions;
	17
	18	fn tmpdir(tag: &str) -> Utf8PathBuf {
	19	static N: AtomicU32 = AtomicU32::new(0);
	20	let n = N.fetch_add(1, Ordering::Relaxed);
	21	let base = Utf8PathBuf::from_path_buf(std::env::temp_dir())
	22	.expect("utf-8 temp dir")
	23	.join(format!("org-ssg-serve-{}-{tag}-{n}", std::process::id()));
	24	let _ = std::fs::remove_dir_all(&base);
	25	std::fs::create_dir_all(&base).unwrap();
	26	base
	27	}
	28
	29	/// An output tree to serve.
	30	fn write_output(out: &Utf8PathBuf) {
	31	std::fs::create_dir_all(out.join("blog")).unwrap();
	32	std::fs::write(out.join("index.html"), "<html><body>home</body></html>").unwrap();
	33	std::fs::write(out.join("blog/index.html"), "<html><body>blog</body></html>").unwrap();
	34	std::fs::write(out.join("syntax.css"), "body{}").unwrap();
	35	std::fs::write(out.join("a file.html"), "<html><body>spaced</body></html>").unwrap();
	36	}
	37
	38	// ---------------------------------------------------------------------------
	39	// URL resolution
	40	// ---------------------------------------------------------------------------
	41
	42	#[test]
	43	fn urls_resolve_to_files_and_directory_indexes() {
	44	let out = tmpdir("resolve");
	45	write_output(&out);
	46
	47	let at = \|url: &str\| resolve(&out, url).map(\|p\| p.strip_prefix(&out).unwrap().to_string());
	48	assert_eq!(at("/"), Some("index.html".into()), "the root serves its index");
	49	assert_eq!(at("/index.html"), Some("index.html".into()));
	50	assert_eq!(at("/blog/"), Some("blog/index.html".into()), "a directory serves its index");
	51	assert_eq!(at("/blog"), Some("blog/index.html".into()), "even without the slash");
	52	assert_eq!(at("/syntax.css"), Some("syntax.css".into()));
	53	assert_eq!(at("/index.html?v=1#frag"), Some("index.html".into()), "query and fragment");
	54	assert_eq!(at("/a%20file.html"), Some("a file.html".into()), "percent-decoded");
	55	assert_eq!(at("/nope.html"), None, "a file that does not exist");
	56	}
	57
	58	/// The one that matters. A dev server sits on a laptop with a home directory behind it.
	59	#[test]
	60	fn no_url_can_escape_the_output_directory() {
	61	let root = tmpdir("traversal");
	62	let out = root.join("out");
	63	write_output(&out);
	64	// A file next to the output that must stay unreachable.
	65	std::fs::write(root.join("secret.txt"), "private").unwrap();
	66
	67	for attack in [
	68	"/../secret.txt",
	69	"/../../etc/passwd",
	70	"/blog/../../secret.txt",
	71	"/%2e%2e/secret.txt",
	72	"/%2E%2E/secret.txt",
	73	"/..%2fsecret.txt",
	74	"/....//secret.txt",
	75	"/\\../secret.txt",
	76	"//../secret.txt",
	77	"/./../secret.txt",
	78	"/blog/%2e%2e/%2e%2e/secret.txt",
	79	] {
	80	assert_eq!(resolve(&out, attack), None, "{attack} must not resolve");
	81	}
	82	// And the file really was reachable by its true path, so the test is not vacuous.
	83	assert!(root.join("secret.txt").is_file());
	84	}
	85
	86	/// A percent-encoded NUL is a classic way to truncate a path in a C-backed API.
	87	#[test]
	88	fn embedded_nul_bytes_are_rejected() {
	89	let out = tmpdir("nul");
	90	write_output(&out);
	91	assert_eq!(resolve(&out, "/index.html%00.txt"), None);
	92	assert_eq!(resolve(&out, "/%00"), None);
	93	}
	94
	95	/// `+` means a space in a query string, not in a path — decoding it would break the
	96	/// perfectly ordinary filename `c++.html`.
	97	#[test]
	98	fn plus_is_not_decoded_as_a_space() {
	99	let out = tmpdir("plus");
	100	write_output(&out);
	101	std::fs::write(out.join("c++.html"), "<html><body>cpp</body></html>").unwrap();
	102	assert_eq!(
	103	resolve(&out, "/c++.html").map(\|p\| p.strip_prefix(&out).unwrap().to_string()),
	104	Some("c++.html".into())
	105	);
	106	}
	107
	108	#[test]
	109	fn the_poll_parameter_is_read_from_the_query() {
	110	assert_eq!(since_parameter("/__org-ssg/reload?since=7"), 7);
	111	assert_eq!(since_parameter("/__org-ssg/reload?x=1&since=42"), 42);
	112	assert_eq!(since_parameter("/__org-ssg/reload"), 0, "absent means start from zero");
	113	assert_eq!(since_parameter("/__org-ssg/reload?since=nope"), 0, "unparseable means zero");
	114	}
	115
	116	// ---------------------------------------------------------------------------
	117	// Reload script injection
	118	// ---------------------------------------------------------------------------
	119
	120	/// The built site is what gets deployed. A dev server's JavaScript must never be in it,
	121	/// which is why injection happens on the way out rather than at build time.
	122	#[test]
	123	fn the_reload_script_is_injected_before_the_closing_body_tag() {
	124	let page = b"<html><body><p>hi</p></body></html>";
	125	let served = String::from_utf8(inject_reload_script(page, 3)).unwrap();
	126
	127	assert!(served.contains("<p>hi</p>"), "content is preserved");
	128	assert!(served.contains("})(3)"), "the generation is baked in: {served}");
	129	let script_at = served.find("<script>").unwrap();
	130	let body_at = served.rfind("</body>").unwrap();
	131	assert!(script_at < body_at, "the script goes inside the body: {served}");
	132	}
	133
	134	/// A fragment with no `</body>` — a partial, or a hand-written page — still gets it.
	135	#[test]
	136	fn injection_falls_back_to_appending() {
	137	let served = String::from_utf8(inject_reload_script(b"<p>bare</p>", 1)).unwrap();
	138	assert!(served.starts_with("<p>bare</p>"));
	139	assert!(served.contains("<script>"));
	140	}
	141
	142	/// Binary content that happens to be served as HTML must not be corrupted into garbage.
	143	#[test]
	144	fn non_utf8_content_is_passed_through_untouched() {
	145	let bytes = vec![0xff, 0xfe, 0x00, 0x42];
	146	assert_eq!(inject_reload_script(&bytes, 1), bytes);
	147	}
	148
	149	// ---------------------------------------------------------------------------
	150	// A live server
	151	// ---------------------------------------------------------------------------
	152
	153	/// Minimal HTTP client: send a request, return the whole response.
	154	fn get(port: u16, path: &str, timeout: Duration) -> Option<String> {
	155	let mut stream = TcpStream::connect(("127.0.0.1", port)).ok()?;
	156	stream.set_read_timeout(Some(timeout)).ok()?;
	157	write!(stream, "GET {path} HTTP/1.1\r\nHost: localhost\r\nConnection: close\r\n\r\n").ok()?;
	158	let mut response = Vec::new();
	159	stream.read_to_end(&mut response).ok()?;
	160	Some(String::from_utf8_lossy(&response).into_owned())
	161	}
	162
	163	fn start_server(src: &Utf8Path, out: &Utf8Path) -> u16 {
	164	// Unique per test as well as per process: these tests run in parallel, and two of
	165	// them sharing a port means one silently queries the other's site.
	166	static NEXT: AtomicU32 = AtomicU32::new(0);
	167	let port = 20000 + ((std::process::id() % 10000) as u16) + NEXT.fetch_add(1, Ordering::Relaxed) as u16;
	168	let (s, o) = (src.to_owned(), out.to_owned());
	169	std::thread::spawn(move \|\| {
	170	let _ = org_ssg::serve::run(&s, &o, &BuildOptions::default(), "127.0.0.1", port);
	171	});
	172	let deadline = Instant::now() + Duration::from_secs(20);
	173	while Instant::now() < deadline {
	174	if get(port, "/", Duration::from_millis(500)).is_some() {
	175	return port;
	176	}
	177	std::thread::sleep(Duration::from_millis(100));
	178	}
	179	panic!("server did not start on port {port}");
	180	}
	181
	182	/// Serve a real site, and confirm an edit both rebuilds and answers a waiting poll —
	183	/// which together are the whole point of the command.
	184	#[test]
	185	fn serving_a_site_reloads_the_browser_when_a_source_changes() {
	186	let root = tmpdir("live");
	187	let src = root.join("src");
	188	std::fs::create_dir_all(&src).unwrap();
	189	std::fs::write(src.join("index.org"), "#+TITLE: Home\n\nFirst version.\n").unwrap();
	190	// Output outside the source, so the test's own writes cannot be mistaken for edits.
	191	let out = root.join("out");
	192	let port = start_server(&src, &out);
	193
	194	let home = get(port, "/", Duration::from_secs(5)).expect("a response");
	195	assert!(home.contains("200 OK"), "{home}");
	196	assert!(home.contains("First version."), "the page is served: {home}");
	197	assert!(home.contains("__org-ssg/reload"), "with the reload script: {home}");
	198	assert!(
	199	!std::fs::read_to_string(out.join("index.html")).unwrap().contains("__org-ssg"),
	200	"but the file on disk stays clean"
	201	);
	202
	203	// A poll for a generation we already have must block, not answer immediately.
	204	let poller = std::thread::spawn(move \|\| {
	205	let started = Instant::now();
	206	let body = get(port, "/__org-ssg/reload?since=0", Duration::from_secs(30));
	207	(started.elapsed(), body)
	208	});
	209	std::thread::sleep(Duration::from_millis(400));
	210	std::fs::write(src.join("index.org"), "#+TITLE: Home\n\nSecond version.\n").unwrap();
	211
	212	let (waited, body) = poller.join().expect("poller");
	213	let body = body.expect("poll response");
	214	assert!(
	215	waited >= Duration::from_millis(300),
	216	"the poll should have waited for the edit, not returned at once ({waited:?})"
	217	);
	218	assert!(body.trim_end().ends_with('1'), "it reports the new generation: {body}");
	219
	220	let updated = get(port, "/", Duration::from_secs(5)).expect("a response");
	221	assert!(updated.contains("Second version."), "and the rebuild is served: {updated}");
	222	}
	223
	224	/// A traversal attempt against the running server, not just the resolver.
	225	#[test]
	226	fn the_running_server_refuses_to_escape_its_root() {
	227	let root = tmpdir("livetraversal");
	228	let src = root.join("src");
	229	std::fs::create_dir_all(&src).unwrap();
	230	std::fs::write(src.join("index.org"), "#+TITLE: Home\n\nBody.\n").unwrap();
	231	std::fs::write(root.join("secret.txt"), "private").unwrap();
	232	let out = root.join("out");
	233	let port = start_server(&src, &out);
	234
	235	for attack in ["/../secret.txt", "/%2e%2e/secret.txt", "/../../etc/passwd"] {
	236	let response = get(port, attack, Duration::from_secs(5)).expect("a response");
	237	assert!(response.contains("404"), "{attack} should 404: {response}");
	238	assert!(!response.contains("private"), "{attack} leaked the file: {response}");
	239	}
	240	}

Commit 20a54d84ca

Commit `20a54d84ca`