Commit dc95376e5c
Verified · cmc
Layout: unified · split
docs/specs/2026-09-11-snippets-design.md added +235
| @@ -0,0 +1,235 @@ | |||
| 1 | # Snippets | ||
| 2 | |||
| 3 | Closes #195 (ref #185). A snippet is a small set of named text files a | ||
| 4 | user owns, shares by URL, and edits in place: paste.sr.ht's paste with a | ||
| 5 | gist's mutability, without the git repository underneath. | ||
| 6 | |||
| 7 | ## Problem | ||
| 8 | |||
| 9 | #185 walked a sourcehut user's day and found that sharing a log or a | ||
| 10 | fragment several times a week has no object here. Neither a repository | ||
| 11 | (too heavy for a log) nor an issue (wrong shape) covers it. #195 asks for | ||
| 12 | the feature rather than a FAQ entry declining it. | ||
| 13 | |||
| 14 | ## Decision | ||
| 15 | |||
| 16 | A snippet is rows in SQLite: an owner, a description, a visibility, and | ||
| 17 | one or more named files with their content. It is created and edited | ||
| 18 | over SSH and the API, rendered and edited on the web through the same | ||
| 19 | control commands, and identified by an opaque id in a URL under the | ||
| 20 | owner. | ||
| 21 | |||
| 22 | Decisions taken on the way, with the alternatives rejected: | ||
| 23 | |||
| 24 | - **Store-backed, not a git repository.** A gist is a bare repository | ||
| 25 | with a flag, cloneable and versioned, which would drag in a namespace | ||
| 26 | decision, a receive-pack path that skips CI, issues and merge | ||
| 27 | requests, and the whole repository policy surface. The use case is a | ||
| 28 | log pasted from a terminal. If history is ever wanted the id and URL | ||
| 29 | scheme below do not have to change. | ||
| 30 | - **Mutable, opaque id.** paste.sr.ht keys a paste on a hash of its | ||
| 31 | content, so a typo fix changes the URL already shared. A random id | ||
| 32 | keeps the URL; updating a file replaces it and no history is kept. | ||
| 33 | - **Three visibilities, default unlisted.** `public` is listed on the | ||
| 34 | owner's page; `unlisted` is readable by anyone with the URL and listed | ||
| 35 | nowhere; `private` is the owner's alone and answers 404 to everyone | ||
| 36 | else, as a private repository does. Sharing a log wants unlisted, | ||
| 37 | so that is the default when `snippet create` names none. | ||
| 38 | - **Users only.** Nothing in #195 or #185 asks for an org to own a | ||
| 39 | snippet, and an org snippet would need a membership rule for writes. | ||
| 40 | - **Web writes in the same merge request.** The Parity rule lands only | ||
| 41 | the triage loop on the web at once; the request here was parity in one | ||
| 42 | go. Every form dispatches a control command through `runControlStdin`, | ||
| 43 | so no rule lives in a handler. | ||
| 44 | - **Text only.** Content must be valid UTF-8. A snippet is read on a | ||
| 45 | page and served raw as `text/plain`; a binary belongs in a release | ||
| 46 | asset. | ||
| 47 | - **One file per command.** Stock OpenSSH carries one stdin stream, so | ||
| 48 | `snippet create` takes one file and `snippet file set` adds the rest. | ||
| 49 | A packed multi-file format on stdin would fail the stock-ssh | ||
| 50 | constraint. | ||
| 51 | - **No `--yes` on delete.** `release delete --yes` guards assets that | ||
| 52 | are gone for good; a snippet has nothing hanging off it and is a | ||
| 53 | paste, not a release. | ||
| 54 | - **No events, audit rows, notifications, comments, search, Atom feed | ||
| 55 | or explore listing.** Events are keyed on a repository. None of the | ||
| 56 | rest was asked for. | ||
| 57 | |||
| 58 | ## Data | ||
| 59 | |||
| 60 | Migration 0053, no rebuild: | ||
| 61 | |||
| 62 | ```sql | ||
| 63 | CREATE TABLE snippets ( | ||
| 64 | id INTEGER PRIMARY KEY, | ||
| 65 | public_id TEXT NOT NULL UNIQUE, | ||
| 66 | owner_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE, | ||
| 67 | description TEXT NOT NULL DEFAULT '', | ||
| 68 | visibility TEXT NOT NULL CHECK (visibility IN ('public','unlisted','private')), | ||
| 69 | created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ','now')), | ||
| 70 | updated_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ','now')) | ||
| 71 | ); | ||
| 72 | CREATE INDEX snippets_owner ON snippets(owner_id, created_at); | ||
| 73 | |||
| 74 | CREATE TABLE snippet_files ( | ||
| 75 | snippet_id INTEGER NOT NULL REFERENCES snippets(id) ON DELETE CASCADE, | ||
| 76 | name TEXT NOT NULL, | ||
| 77 | content BLOB NOT NULL, | ||
| 78 | size INTEGER NOT NULL, | ||
| 79 | PRIMARY KEY (snippet_id, name) | ||
| 80 | ); | ||
| 81 | ``` | ||
| 82 | |||
| 83 | `public_id` is 12 lowercase hex characters from `crypto/rand` (48 bits), | ||
| 84 | generated on create and retried on a unique violation. It is global: the | ||
| 85 | CLI takes `<id>` alone, and the owner in the URL is for reading, not for | ||
| 86 | lookup. | ||
| 87 | |||
| 88 | `updated_at` moves on every file or metadata write. Deleting a user | ||
| 89 | deletes their snippets through the cascade, as it does their keys. | ||
| 90 | |||
| 91 | Limits: | ||
| 92 | |||
| 93 | - `limits.max_snippet_bytes` in `config.toml`, per file, default | ||
| 94 | 1 MiB. Enforced on create and `file set` with the same | ||
| 95 | `io.LimitReader(n+1)` shape as `release asset add`. | ||
| 96 | - 64 files per snippet, a constant in `internal/control/snippet.go`. | ||
| 97 | - Snippet bytes do not count toward `max_bytes_per_user`; that quota | ||
| 98 | measures repositories and LFS. | ||
| 99 | |||
| 100 | File names match the release asset name pattern: | ||
| 101 | `^[A-Za-z0-9][A-Za-z0-9._+-]{0,199}$`, so no slashes and no leading dot. | ||
| 102 | |||
| 103 | `store.Snippet` carries the row plus `OwnerName`; `store.SnippetFile` | ||
| 104 | carries `Name`, `Size` and `Content`. Store functions: `CreateSnippet`, | ||
| 105 | `SnippetByPublicID`, `ListSnippets(ownerID, all bool, limit, cursor)`, | ||
| 106 | `UpdateSnippet`, `DeleteSnippet`, `SetSnippetFile`, `RemoveSnippetFile`, | ||
| 107 | `SnippetFiles`, `SnippetFile`. Hand-written SQL, as everywhere. | ||
| 108 | |||
| 109 | ## Commands | ||
| 110 | |||
| 111 | All in `internal/control/snippet.go`, registered like every other noun. | ||
| 112 | Reads set `ReadOnly`; the two commands that take a body set | ||
| 113 | `ReadsStdin`; nothing is `SSHOnly`, so the JSON API reaches all of it. | ||
| 114 | |||
| 115 | | command | usage | | ||
| 116 | |---|---| | ||
| 117 | | `snippet create` | `snippet create <filename> [--description <d>] [--visibility public\|unlisted\|private] < file` | | ||
| 118 | | `snippet show` | `snippet show <id>` | | ||
| 119 | | `snippet list` | `snippet list [<owner>] [--limit n] [--cursor c]` | | ||
| 120 | | `snippet edit` | `snippet edit <id> [--description <d>] [--visibility <v>]` | | ||
| 121 | | `snippet delete` | `snippet delete <id>` | | ||
| 122 | | `snippet file set` | `snippet file set <id> <filename> < file` | | ||
| 123 | | `snippet file get` | `snippet file get <id> <filename> > file` | | ||
| 124 | | `snippet file remove` | `snippet file remove <id> <filename>` | | ||
| 125 | |||
| 126 | Behaviour: | ||
| 127 | |||
| 128 | - `create` reads stdin as the first file, refuses empty stdin, content | ||
| 129 | that is not valid UTF-8, or content over the limit (exit 2 with the | ||
| 130 | reason), and prints the id and the web URL. JSON: `{id, url, owner, | ||
| 131 | description, visibility, created_at, updated_at, files:[{name,size}]}`. | ||
| 132 | - `show` prints the metadata and the file list with sizes. `--json` | ||
| 133 | includes each file's `content` so one API read returns the whole | ||
| 134 | snippet. | ||
| 135 | - `list` with no argument lists the caller's snippets at every | ||
| 136 | visibility, newest first. With `<owner>` it lists that owner's public | ||
| 137 | snippets, or everything when the owner is the caller or an admin. An | ||
| 138 | unknown owner is exit 3. Paged with keyset cursors like `repo list`; | ||
| 139 | the JSON is `{items, next}`. | ||
| 140 | - `edit` changes the description or the visibility, or both; neither | ||
| 141 | given is exit 2. | ||
| 142 | - `file set` adds a file or replaces one by name, under the same checks | ||
| 143 | as `create`, and refuses the 65th file. `file remove` refuses to | ||
| 144 | remove the last file: a snippet always has one. `file get` writes the | ||
| 145 | content to stdout unchanged. | ||
| 146 | - `delete` removes the snippet and its files. | ||
| 147 | |||
| 148 | Access, in `internal/policy` beside the repository rules. Key scope needs | ||
| 149 | no rule of its own: the dispatcher already refuses every control command | ||
| 150 | to a key that is not `full`. | ||
| 151 | |||
| 152 | - Read: the owner and admins for `private`; anyone, anonymous included, | ||
| 153 | for `unlisted` and `public`. | ||
| 154 | - Write: the owner and admins. | ||
| 155 | - A snippet the caller may not read is exit 3, never 4, so a private id | ||
| 156 | cannot be confirmed. A snippet the caller may read but not write is | ||
| 157 | exit 4. | ||
| 158 | |||
| 159 | CLI: one `pass()` per command in `cmd/gitbay/main.go`. `snippet create` | ||
| 160 | and `snippet file set` use `alwaysStdin` with a `stdinWhat` naming the | ||
| 161 | file's bytes, as `release asset add` does. No `needsRepo`: the noun is | ||
| 162 | not repository-scoped, and the repo argument is never inferred. | ||
| 163 | |||
| 164 | ## Web | ||
| 165 | |||
| 166 | Routes live under `/{owner}/-/`, the pattern `/{owner}/-/labels` set: a | ||
| 167 | hyphen cannot start a repository name, so nothing is shadowed and no | ||
| 168 | word joins `internal/policy/names.go`. | ||
| 169 | |||
| 170 | | method | path | handler | | ||
| 171 | |---|---|---| | ||
| 172 | | GET | `/{owner}/-/snippets` | list: the owner's public snippets; everything, marked by visibility, when the viewer is the owner or an admin | | ||
| 173 | | GET | `/{owner}/-/snippets/new` | create form, `requireUser`, owner must be the viewer | | ||
| 174 | | POST | `/{owner}/-/snippets/new` | dispatch `snippet create`, redirect to the snippet | | ||
| 175 | | GET | `/{owner}/-/snippets/{id}` | the snippet: description, visibility, each file highlighted with a raw link | | ||
| 176 | | GET | `/{owner}/-/snippets/{id}/raw/{name}` | `text/plain; charset=utf-8`, `X-Content-Type-Options: nosniff` | | ||
| 177 | | POST | `/{owner}/-/snippets/{id}/edit` | dispatch `snippet edit` | | ||
| 178 | | POST | `/{owner}/-/snippets/{id}/delete` | dispatch `snippet delete`, redirect to the list | | ||
| 179 | | POST | `/{owner}/-/snippets/{id}/file` | dispatch `snippet file set` with the textarea on stdin | | ||
| 180 | | POST | `/{owner}/-/snippets/{id}/file/remove` | dispatch `snippet file remove` | | ||
| 181 | |||
| 182 | An id under the wrong owner is 404. Private snippets are 404 to anyone | ||
| 183 | but the owner and admins, unlisted ones render for anyone with the URL. | ||
| 184 | Files are rendered through the existing `highlight(path, data)` by | ||
| 185 | extension; `.md` and `.org` are highlighted as source, not rendered as | ||
| 186 | markup, since a snippet is a paste rather than a document. Each file | ||
| 187 | heading links to its raw route. | ||
| 188 | |||
| 189 | Forms are on the snippet page for the owner: a textarea per file posting | ||
| 190 | `file`, a remove button per file, an add-file form (name and textarea), | ||
| 191 | a description and visibility form, and delete. All POSTs go through | ||
| 192 | `checkOrigin` and `requireUser`, and answer through `done`, so a refusal | ||
| 193 | returns to the page with the message and a missing snippet is the 404 | ||
| 194 | page. | ||
| 195 | |||
| 196 | The owner page shows a `snippets` link beside the repositories heading | ||
| 197 | when the owner has a public snippet, or when the viewer is the owner. | ||
| 198 | |||
| 199 | Templates: `snippets.html`, `snippet.html`, `snippetnew.html`. Stylesheet | ||
| 200 | additions in `static/style.css` only where an existing class does not | ||
| 201 | fit; the file blocks reuse the blob page's classes. | ||
| 202 | |||
| 203 | ## Testing | ||
| 204 | |||
| 205 | `e2e/snippet_test.go`, over ssh with the real binary: | ||
| 206 | |||
| 207 | - create prints an id of 12 hex characters and a URL; `show` and `file | ||
| 208 | get` round-trip the content byte for byte; `--json` on `show` carries | ||
| 209 | `content`. | ||
| 210 | - visibility, from a second account and from anonymous HTTP: private is | ||
| 211 | exit 3 and 404 to the other account, unlisted is readable by id and | ||
| 212 | absent from `snippet list <owner>`, public is listed; the owner's own | ||
| 213 | `snippet list` shows all three. | ||
| 214 | - `file set` replaces, `file remove` refuses the last file, the 65th | ||
| 215 | file is refused, non-UTF-8 and oversize bodies are refused with exit 2. | ||
| 216 | - `edit` moves visibility and the listing follows. | ||
| 217 | - `delete` then `show` is exit 3; deleting the user cascades the rows. | ||
| 218 | - the other account cannot `edit`, `file set` or `delete` (exit 4 on an | ||
| 219 | unlisted snippet, exit 3 on a private one). | ||
| 220 | |||
| 221 | `e2e/snippetweb_test.go`: the list and snippet pages render, raw is | ||
| 222 | `text/plain` with nosniff, the create form makes a snippet, the file and | ||
| 223 | edit forms change it, delete removes it, and the owner page carries the | ||
| 224 | link. The existing coverage tests hold the rest: the CLI table, the | ||
| 225 | `ReadsStdin` flag, and that every `ReadOnly` command writes nothing. | ||
| 226 | |||
| 227 | ## Documentation | ||
| 228 | |||
| 229 | - `.gitbay/wiki/Users.org`: a `* Snippets` section after Pages. | ||
| 230 | - `.gitbay/wiki/Parity.org`: rows for `snippet create, edit, delete`, | ||
| 231 | `snippet show, list`, `snippet file set, get, remove`; `cli` yes, `web` | ||
| 232 | yes, `ios` no. | ||
| 233 | - `CHANGELOG.org`: the v1.20.0 entry. | ||
| 234 | - `.gitbay/wiki/Admin.org`: `max_snippet_bytes` beside `max_asset_bytes` | ||
| 235 | in the limits list. | ||