README.md
311 lines · 16231 bytes
1# ambient-companions
2
3*Five faces over one backend.*
4
5One signal-collection daemon; five thin renderers that subscribe to it. This is
6closer to **one project with five faces** than five projects: the backend is
7the project, and each face is thin, disposable, and deletable without touching
8the others.
9
10## One backend, five faces
11
12```text
13 signald (user LaunchAgent)
14 zsh hooks ───▶ terminal collector ─┐
15 git/fsevents ▶ git collector ──────┼─▶ normalizer ─▶ ring buffer
16 IOKit/AppKit ▶ system+hw collector ┘ │
17 ▼
18 publish: Unix socket + SQLite WAL (history)
19 ┌───────────┬───────────┼───────────┬───────────────┐
20 ▼ ▼ ▼ ▼ ▼
21 terminal-pet garden menubar-pet sonifier wallpaperd
22 (TUI) (TUI) (SwiftUI) (AVAudio) (image/window)
23```
24
25All five draw from the **same signal bus**. Renderers never poll hardware,
26never read the shell — they subscribe over a local Unix socket
27(`$XDG_RUNTIME_DIR/signald.sock`). That buys one privacy boundary to defend
28instead of five, cheap renderers, and reuse of the collection layer into
29non-toy outputs (e-ink dashboards, printed posters).
30
31## The privacy boundary, enforced by construction
32
33Reading typing during real work is a keylogger unless it is *structurally*
34content-free — and unless that is a **test**, not a sentence in a README.
35
36The invariant:
37
38> No process persists, transmits, or exposes any representation from which the
39> content or identity of an individual keystroke, command argument, or typed
40> character can be recovered. Only order-free aggregates (counts, rates,
41> durations, codes) leave the terminal collector.
42
43Made true by construction:
44
45- **No content channel exists in the wire format.** The only payload channel is
46 `Value`, a newtype over `f64`. There is no `text`, `bytes`, or `payload`
47 field — a collector *cannot* emit typed content because the record has
48 nowhere to put it. The one audited exception is a non-content identifier in
49 `tag` — an absolute repo path, allow-listed to the four git metric names,
50 and confined by the daemon to the roots it was told to watch.
51- **The key counter never stores the key**, there is **no input tap anywhere**
52 (`CGEventTap`, `IOHIDManager` keyboard usage, accessibility observation are
53 all forbidden), and **aggregation happens before transport** (the shell emits
54 counts on `precmd`, never per-key events).
55
56And it is tested (`crates/signal-schema/tests/privacy_invariant.rs`):
57
58- `value_channel_is_exactly_f64` — the payload is an `f64`, nothing wider.
59- `wire_format_has_no_content_field` — the `Signal` type declares no
60 content-carrying field beyond the audited `tag`.
61- `forbidden_symbol_scan` — the tree contains none of the banned keylogger APIs
62 or shell line-buffer references (the static gate).
63- `differential_secret_typing` — **active (the ship gate).** It drives the real
64 hook (`shell-hooks/signald-hooks.zsh`) through a real interactive zsh under a
65 real pseudo-terminal (zsh's own `zsh/zpty` — no extra dependency), *typing a
66 planted secret*, and asserts the secret never appears — plain, reversed, hex,
67 or base64 — in the shell spool or the `f64`-only wire encoding. The **full
68 pipeline** version (same real typing driven through the terminal collector,
69 the SQLite history store, and the hub) is
70 `crates/signald/tests/differential_secret_typing.rs`. Real `zle` keystroke
71 counting increments a *number* per key and discards the key, so the only thing
72 the pipeline ever receives about the typing is a count — the tests prove that
73 empirically across every downstream artifact.
74
75## Build order
76
77Build the backend to the point each renderer needs, then build the renderer
78that is cheapest *and* most valuable given what exists:
79
801. **garden** — git is the cleanest signal (discrete, no privacy questions), so
81 it proves the bus first. *(recommended first face)*
822. **gated collector + pet** — add the sensitive terminal collector only after
83 the privacy tests are green and extended to cover it.
843. **menu-bar permadeath pet** — first macOS-native face; ships the one-life
85 variant (ages in wall-clock time, dies permanently) — the emotional hook.
864. **SSH-sonification** — unexpected access becomes *audible*; the utility
87 sonification before the ambient one.
885. **homelab wallpaper** — desktop-as-status-board (Path A: render-to-image +
89 `setDesktopImageURL`); the same frame pipeline later feeds e-ink and posters.
90
91## macOS collector constraint
92
93The system + hardware collector is **IOKit-only: no root, no `powermetrics`.**
94`powermetrics` wants root and would make the suite un-shippable as a plain user
95agent. Signals come from IOKit (`IOPMPowerSource` / power sources, IORegistry
96`AppleSmartBattery`), `ProcessInfo`, and Mach `host_processor_info`. Where a
97metric can't be reached without root, it is simply **absent from the schema**
98rather than gated behind sudo. Accessibility permission is never requested.
99
100It ships as **`macos-collector/`**, a sibling **Swift** package (built with
101`swift build`, kept out of the cargo workspace — SwiftPM and cargo do not share
102a build system). It reads aggregate CPU load, battery %, charging, battery draw
103(W), and thermal state, and emits them as `signal-schema` wire frames — the
104same byte format `signald` parses. `signald` spawns it as a child process
105(found on `PATH`, or named with `--collector <path>`) and ingests the frames it
106writes to stdout, so hardware signals reach the hub, the history store, and
107every subscriber by the same path as git and terminal signals. GPU/fan are
108deliberately omitted (no clean root-free IOKit channel). See `macos-collector/README.md` for the full **Swift ↔
109Rust wire contract** (the byte layout) and the shared canonical-frame test that
110pins both sides to the same bytes.
111
112## Workspace layout
113
114```text
115ambient-companions/
116├── Cargo.toml # Rust workspace
117├── crates/
118│ ├── signal-schema/ # shared wire format (the privacy boundary)
119│ │ ├── src/lib.rs
120│ │ └── tests/privacy_invariant.rs
121│ ├── signald/ # the daemon (collectors + live fan-out)
122│ │ ├── src/lib.rs # git + terminal collectors, hardware ingest, publish
123│ │ ├── src/history.rs # SQLite (WAL) history store + recent() query
124│ │ ├── src/hub.rs # last-value cache + live fan-out
125│ │ ├── src/main.rs # CLI, producer loop, self-attestation
126│ │ ├── tests/git_collector.rs # aggregates vs a temp git repo
127│ │ ├── tests/streaming.rs # last-value cache + live update
128│ │ ├── tests/hardware_ingest.rs # collector frames reach a subscriber
129│ │ └── tests/differential_secret_typing.rs # the full privacy ship-gate
130│ └── terminal-garden/ # first renderer: a socket subscriber
131│ ├── src/lib.rs # signals → plots → render (unit-tested)
132│ └── src/main.rs # live subscribe + redraw loop
133├── shell-hooks/ # zsh hooks: aggregate-only terminal collector
134│ ├── signald-hooks.zsh
135│ └── README.md
136└── macos-collector/ # SwiftPM sibling (NOT in the cargo workspace):
137 ├── Package.swift # macOS IOKit hardware collector (Phase 3)
138 ├── README.md # the Swift↔Rust wire contract (byte layout)
139 ├── Sources/CollectorCore/ # wire encoder + IOKit reads (host_processor_
140 │ # info, power sources, AppleSmartBattery)
141 ├── Sources/macos-collector/ # thin CLI: --once / stream / --hex / --out
142 └── Tests/ # shared canonical-frame wire-contract test
143```
144
145### Language note
146
147The daemon is written in **Rust**, chosen
148because it makes the "no content field exists" guarantee enforceable in the
149type system (the `f64`-only `Value` payload), which is the whole point of the
150privacy boundary.
151
152### Dependencies
153
154`signal-schema` is **dependency-free** by design — the wire format is the
155privacy boundary and carries no third-party code. `signald` has **one**
156dependency, `rusqlite` (with the `bundled` feature, so SQLite is compiled
157in-tree and there is no system-library requirement), for the WAL history store.
158The terminal collector's shell side needs only `zsh` (the `zle`, `datetime`,
159and — for the differential test — `zpty` modules ship with zsh).
160
161## Phase plan
162
163- **M0 — backend skeleton + schema + privacy invariant test. ✅ Done.**
164 Workspace builds; the structural privacy tests pass.
165- **Phase 1 — git collector + garden. ✅ Done (v0.1).**
166 The wire format (`encode`/`decode` + length-prefixed framing) is real and
167 `f64`-only. `signald`'s git collector shells out to `git` for aggregate
168 scalars — commits-in-window, commits-today, branch count, days-since-last-
169 commit — tagged by the audited repo path. `terminal-garden` subscribes and
170 renders each repo as a plot: growth (🌱→🌿→🌳) tracks recent commits, wilt
171 (🥀→🍂) tracks staleness.
172- **Persistence + live streaming. ✅ Done (v0.2).**
173 - **SQLite (WAL) history** (`signald/src/history.rs`): every published signal
174 is persisted (aggregate scalars only — same privacy constraints as the
175 wire), and `recent()` / `recent_named()` let a renderer read recent history,
176 not just the live snapshot.
177 - **Live streaming + last-value cache** (`signald/src/hub.rs`,
178 `publish::serve`): the daemon runs a producer loop and continuously
179 publishes. On connect a subscriber is handed the current value of every
180 metric immediately, then streams updates as they change (no longer
181 snapshot-then-close). `terminal-garden` now redraws live.
182- **Phase 2 — terminal collector. ✅ Done (v0.2).**
183 The aggregate-only terminal path is real: `shell-hooks/signald-hooks.zsh`
184 counts keystrokes with a `zle` widget that increments a number and discards
185 the key, and appends `<epoch_ms> <keys> <session_seconds> <session_id>` count
186 records (numbers only) to a spool; `collectors::terminal` consumes the spool
187 and derives `keys_per_min` and `session_seconds`. The **differential
188 secret-typing test is active and passing** — the privacy ship-gate. *The
189 Terminal Pet renderer is still todo.*
190- **Phase 3 — macOS IOKit hardware collector. ✅ Done (v0.3).**
191 `macos-collector/` (a sibling Swift package, `swift build`) reads aggregate
192 hardware scalars via **IOKit only — no `powermetrics`, no root**: CPU load
193 (Mach `host_processor_info`), battery %/charging (IOKit power sources), battery
194 draw in watts (IORegistry `AppleSmartBattery`), and thermal state
195 (`ProcessInfo`). It emits `signal-schema` wire frames — the schema gained the
196 aggregate `cpu_load` metric and bumped to `SCHEMA_VERSION = 3`. GPU/fan are
197 omitted (no clean root-free channel). The **Swift ↔ Rust byte contract** is
198 documented and pinned by a shared canonical-frame test on both sides
199 (`crates/signal-schema/tests/hardware_wire.rs` decodes the exact bytes the
200 Swift encoder commits to in `macos-collector/Tests/.../WireTests.swift`).
201 *Still todo:* the menu-bar permadeath pet.
202- **Hardware ingest. ✅ Done (v0.4).**
203 `signald` spawns `macos-collector` as a child (`--collector <path>`, or found
204 on `PATH`) and reads its stdout with the same `wire::read_frame` the socket
205 uses (`collectors::hardware`). The five hardware signals now appear in the
206 hub, the history store, and every subscriber's snapshot
207 (`crates/signald/tests/hardware_ingest.rs`).
208- **Bounded spool and history. ✅ Done (v0.4).**
209 The terminal spool is consumed each tick (renamed aside, read, deleted)
210 instead of re-read in full forever. Records carry the shell's pid, so
211 `keys_per_min` is each active shell's rate summed rather than a mix of
212 interleaved sessions, and `session_seconds` is the longest active shell.
213 History rows older than `--retention-days` (default 7) are pruned on open
214 and every 1000 inserts.
215- **Contract freeze. ✅ Done (v0.5).**
216 `SignalName` is cut to the eleven metrics that have a producer, renumbered
217 from zero, and `SCHEMA_VERSION` is **4 — the 1.0 contract**. Discriminants
218 are append-only from 1.0, so v4 was the last chance to renumber. The only
219 audited tag identifier left is the repo path, confined by the daemon to the
220 roots it was told to watch. A reader now **skips** a frame it cannot decode
221 instead of dying on it (`wire::Frame::Skipped`), so an older renderer keeps
222 working against a newer daemon. References to an uncommitted spec are gone.
223- **Phase 4** — sonification (SSH-utility first, then continuous). *Out of scope.*
224- **Phase 5** — live wallpaper (homelab, Path A) + e-ink/poster reuse.
225 *Out of scope.*
226
227## Install
228
229```sh
230brew install krz/tap/ambient-companions
231brew services start ambient-companions
232```
233
234Name the repositories to watch — signald takes none by default, and a login
235agent has no useful working directory:
236
237```sh
238mkdir -p ~/.config/signald
239cat > ~/.config/signald/repos <<'EOF'
240# one repository path per line; # comments and blank lines are ignored
241~/git/some-repo
242EOF
243brew services restart ambient-companions
244```
245
246Then add one line to `.zshrc` for the terminal collector:
247
248```sh
249source "$(brew --prefix)/share/ambient-companions/signald-hooks.zsh"
250```
251
252Open a new shell and watch the garden:
253
254```sh
255terminal-garden
256```
257
258`brew services` logs to `$(brew --prefix)/var/log/`. To run the agent by hand
259instead, `packaging/net.krz.signald.plist` is a launchd template; its header
260comment carries the `sed` line that fills in the paths and the `launchctl load`
261that starts it.
262
263### Paths
264
265Defaults, resolved the same way by `signald`, `terminal-garden`, and
266`shell-hooks/signald-hooks.zsh`. This table is the single place they are
267written down; unit tests in both binaries pin the code to it.
268
269| | `$XDG_RUNTIME_DIR` set | otherwise |
270|---|---|---|
271| socket | `$XDG_RUNTIME_DIR/signald.sock` | `~/.local/state/signald/sock` |
272| history db | `$XDG_RUNTIME_DIR/signald.sqlite` | `~/.local/state/signald/signald.sqlite` |
273| terminal spool | `$XDG_RUNTIME_DIR/terminal.spool` | `~/.local/state/signald/terminal.spool` |
274
275The db and the spool are derived from the socket's directory, so `--socket`
276moves all three together. Override individually with `--db` and `--spool`, and
277the spool from the shell side with `$SIGNALD_SPOOL` — it must match whatever
278the daemon uses, or the terminal metrics stay silently empty. launchd sets no
279`XDG_RUNTIME_DIR`, so an installed agent lands in `~/.local/state/signald`.
280
281The repository list is `$XDG_CONFIG_HOME/signald/repos`, else
282`~/.config/signald/repos`. Positional arguments to `signald` override it; with
283neither, the working directory is watched.
284
285Unix socket paths are limited to about 104 bytes. A deep scratch directory
286will hit `SUN_LEN`; use `mktemp -d` when testing by hand.
287
288## Build & test
289
290```sh
291cargo build # whole Rust workspace
292cargo test # includes the privacy invariant suite (must be green)
293
294# The macOS IOKit hardware collector is a sibling Swift package (macOS only):
295cd macos-collector
296swift build # builds the collector
297swift test # the Swift↔Rust wire-contract test
298swift run macos-collector --once # one real IOKit read (no root)
299```
300
301### Pre-push checks
302
303There is no CI. `.githooks/pre-push` is the gate: it runs `cargo test
304--locked`, `cargo clippy --all-targets --locked -- -D warnings`, and the
305`macos-collector` Swift tests, and a failure aborts the push. The differential
306secret-typing tests need `zsh` with the `zsh/zpty` module; the hook checks for
307it first and fails rather than skipping the gate. Enable it once per clone:
308
309```sh
310git config core.hooksPath .githooks
311```