Commit 5c69b0125f
Verified · cmc ci/build: success ci/lint: success ci/test: success
Layout: unified · split
ROADMAP.md +92 −4
| @@ -15,10 +15,13 @@ below where they overlap. | |||
| 15 | 15 | ||
| 16 | ## Status | 16 | ## Status |
| 17 | 17 | ||
| 18 | Everything below except 5.3 and 5.5 shipped in v1.5.0 (2026-09-11), with | 18 | Everything below except 5.3 shipped in v1.5.0 (2026-09-11), with v1.5.1 |
| 19 | v1.5.1 fixing the release build. Merge requests !6 through !25 on gitbay. | 19 | fixing the release build. Six patch releases followed on 2026-09-11 and |
| 20 | Still open: #33 (LibRedirect submission), #1, #2, #3 and #6 from the | 20 | 2026-09-12 from running the public instance; see "After v1.5.0" at the |
| 21 | original list, which need real DeviantArt payloads to work from. | 21 | end. Of the original six issues, #1, #3, #4, #5 and #6 are closed by |
| 22 | merged work and #2 is closed as not possible for a guest session. The | ||
| 23 | one open issue is #33, the LibRedirect submission, which needs the | ||
| 24 | maintainer's account. | ||
| 22 | 25 | ||
| 23 | ## Ordering principle | 26 | ## Ordering principle |
| 24 | 27 | ||
| @@ -376,3 +379,88 @@ Two things the stack taught: lint on macOS never compiles the Linux-only | |||
| 376 | files, so run `GOOS=linux golangci-lint run` before pushing; and `go get` | 379 | files, so run `GOOS=linux golangci-lint run` before pushing; and `go get` |
| 377 | can raise the go directive in go.mod, so check the Dockerfile and | 380 | can raise the go directive in go.mod, so check the Dockerfile and |
| 378 | workflow images still match it. | 381 | workflow images still match it. |
| 382 | |||
| 383 | ## After v1.5.0 | ||
| 384 | |||
| 385 | Everything here came out of deploying v1.5 to art.krz.sh and watching it. | ||
| 386 | |||
| 387 | ### v1.5.1 (!24, !25) | ||
| 388 | |||
| 389 | go.mod had moved to Go 1.26 when x/sync came in while the Dockerfile and | ||
| 390 | the binaries job still used 1.25, so the v1.5.0 tag built no image. Both | ||
| 391 | now match go.mod. The tarball job also failed on VCS stamping inside the | ||
| 392 | build container, fixed with `-buildvcs=false`. | ||
| 393 | |||
| 394 | ### v1.5.2 (!27, !28) | ||
| 395 | |||
| 396 | The instance's VPN exit was banned by DeviantArt's WAF and every | ||
| 397 | DeviantArt-backed page 502d until someone restarted the stack. | ||
| 398 | |||
| 399 | - API cache entries past their TTL are kept for `api-cache.stale` | ||
| 400 | (default 1h) and served when upstream fails or answers 403 or 429. A | ||
| 401 | block starts a one minute backoff during which the instance stops | ||
| 402 | asking. | ||
| 403 | - `/api/random` answered 401 with proxying on: the media signing token | ||
| 404 | was inside the path. Fixed, and `Download` sits behind a seam. | ||
| 405 | - The session bootstrap runs before the listener opens, so the first | ||
| 406 | seconds after a restart no longer 502. | ||
| 407 | |||
| 408 | Operationally: the instance moved off the VPN to its own address, which | ||
| 409 | was clean at the time, and the container got a real log driver (it had | ||
| 410 | `none`, which hid every error line). | ||
| 411 | |||
| 412 | ### v1.5.3 (!29) | ||
| 413 | |||
| 414 | The direct address was banned within an hour. `upstream.min-interval-ms` | ||
| 415 | and `upstream.max-concurrent` replace the source constants; the | ||
| 416 | instance runs at 1000 ms and one in flight. The ban lifted after about | ||
| 417 | seventy minutes. The instance also raised `api-cache.ttl` to 30 | ||
| 418 | minutes and `stale` to a day, and a Cloudflare managed-challenge rule | ||
| 419 | now covers non-browser clients on search, post and profile paths. | ||
| 420 | |||
| 421 | ### v1.5.4 (!30, !31) | ||
| 422 | |||
| 423 | Cache rotation trims the oldest files down to `max-size` instead of | ||
| 424 | emptying the directory, and never touches the directory itself, which | ||
| 425 | on a bind mount logged `unlinkat` and `mkdir` errors every pass (#36). | ||
| 426 | The per-platform stat files went with it. x/net bumped (#35). | ||
| 427 | |||
| 428 | ### v1.5.5 (!32) | ||
| 429 | |||
| 430 | A crawler walking post pages at 70 a minute produced 733 upstream | ||
| 431 | timeouts in ten minutes while the address stayed unbanned: each queued | ||
| 432 | request still took its interval turn after its client had timed out. | ||
| 433 | The throttle now honours the request context, and sheds a request that | ||
| 434 | would queue longer than 20 seconds with a 503 and `Retry-After`. The | ||
| 435 | instance's `rate-limit` dropped to 20 per minute, burst 10. | ||
| 436 | |||
| 437 | ### v1.5.6 (!33, !34) | ||
| 438 | |||
| 439 | The real cause of #3: DeviantArt's editor stores descriptions and | ||
| 440 | comments as a document tree (`{"version":1,"document":...}`), not | ||
| 441 | Draft.js blocks, so every current description rendered empty. A new | ||
| 442 | renderer covers the node set seen in 92 live payloads: paragraphs, | ||
| 443 | headings, lists, quotes, code, breaks, rules, text marks, emotes (#6), | ||
| 444 | embedded artworks, GIF embeds and mentions. Media and post URLs are | ||
| 445 | parsed instead of sliced by offset (#1), and old HTML emotes map by | ||
| 446 | image name. | ||
| 447 | |||
| 448 | ### Closed without code | ||
| 449 | |||
| 450 | #2 search filters: DeviantArt's guest search ignores every `order` | ||
| 451 | value the site itself uses, and the deviations endpoint redirects | ||
| 452 | guests. Nothing to expose. #4 and #5 were covered by the instance | ||
| 453 | checker and the Makefile. | ||
| 454 | |||
| 455 | ### Lessons | ||
| 456 | |||
| 457 | - DeviantArt bans an address on volume, not on whether it is a VPN. | ||
| 458 | Pacing, caching and shedding at the instance are what keep it clean; | ||
| 459 | rotating exits only buys an hour. | ||
| 460 | - A restart empties the in-memory API cache, so a ban right after a | ||
| 461 | deploy has nothing stale to serve. Deploy when the instance is quiet. | ||
| 462 | - Log driver `none` is a trap. Every incident here was diagnosed from | ||
| 463 | lines that driver would have dropped. | ||
| 464 | - Fetch real payloads from a host DeviantArt accepts before touching a | ||
| 465 | parser; the format had changed under the old one. | ||
| 466 | |||