Commit 5c69b0125f
Verified · cmc ci/build: success ci/lint: success ci/test: success
Layout: unified · split
ROADMAP.md +92 −4
| @@ -15,10 +15,13 @@ below where they overlap. | ||
| 15 | 15 | |
| 16 | 16 | ## Status |
| 17 | 17 | |
| 18 | Everything below except 5.3 and 5.5 shipped in v1.5.0 (2026-09-11), with | |
| 19 | v1.5.1 fixing the release build. Merge requests !6 through !25 on gitbay. | |
| 20 | Still open: #33 (LibRedirect submission), #1, #2, #3 and #6 from the | |
| 21 | original list, which need real DeviantArt payloads to work from. | |
| 18 | Everything below except 5.3 shipped in v1.5.0 (2026-09-11), with v1.5.1 | |
| 19 | fixing the release build. Six patch releases followed on 2026-09-11 and | |
| 20 | 2026-09-12 from running the public instance; see "After v1.5.0" at the | |
| 21 | end. Of the original six issues, #1, #3, #4, #5 and #6 are closed by | |
| 22 | merged work and #2 is closed as not possible for a guest session. The | |
| 23 | one open issue is #33, the LibRedirect submission, which needs the | |
| 24 | maintainer's account. | |
| 22 | 25 | |
| 23 | 26 | ## Ordering principle |
| 24 | 27 | |
| @@ -376,3 +379,88 @@ Two things the stack taught: lint on macOS never compiles the Linux-only | ||
| 376 | 379 | files, so run `GOOS=linux golangci-lint run` before pushing; and `go get` |
| 377 | 380 | can raise the go directive in go.mod, so check the Dockerfile and |
| 378 | 381 | workflow images still match it. |
| 382 | ||
| 383 | ## After v1.5.0 | |
| 384 | ||
| 385 | Everything here came out of deploying v1.5 to art.krz.sh and watching it. | |
| 386 | ||
| 387 | ### v1.5.1 (!24, !25) | |
| 388 | ||
| 389 | go.mod had moved to Go 1.26 when x/sync came in while the Dockerfile and | |
| 390 | the binaries job still used 1.25, so the v1.5.0 tag built no image. Both | |
| 391 | now match go.mod. The tarball job also failed on VCS stamping inside the | |
| 392 | build container, fixed with `-buildvcs=false`. | |
| 393 | ||
| 394 | ### v1.5.2 (!27, !28) | |
| 395 | ||
| 396 | The instance's VPN exit was banned by DeviantArt's WAF and every | |
| 397 | DeviantArt-backed page 502d until someone restarted the stack. | |
| 398 | ||
| 399 | - API cache entries past their TTL are kept for `api-cache.stale` | |
| 400 | (default 1h) and served when upstream fails or answers 403 or 429. A | |
| 401 | block starts a one minute backoff during which the instance stops | |
| 402 | asking. | |
| 403 | - `/api/random` answered 401 with proxying on: the media signing token | |
| 404 | was inside the path. Fixed, and `Download` sits behind a seam. | |
| 405 | - The session bootstrap runs before the listener opens, so the first | |
| 406 | seconds after a restart no longer 502. | |
| 407 | ||
| 408 | Operationally: the instance moved off the VPN to its own address, which | |
| 409 | was clean at the time, and the container got a real log driver (it had | |
| 410 | `none`, which hid every error line). | |
| 411 | ||
| 412 | ### v1.5.3 (!29) | |
| 413 | ||
| 414 | The direct address was banned within an hour. `upstream.min-interval-ms` | |
| 415 | and `upstream.max-concurrent` replace the source constants; the | |
| 416 | instance runs at 1000 ms and one in flight. The ban lifted after about | |
| 417 | seventy minutes. The instance also raised `api-cache.ttl` to 30 | |
| 418 | minutes and `stale` to a day, and a Cloudflare managed-challenge rule | |
| 419 | now covers non-browser clients on search, post and profile paths. | |
| 420 | ||
| 421 | ### v1.5.4 (!30, !31) | |
| 422 | ||
| 423 | Cache rotation trims the oldest files down to `max-size` instead of | |
| 424 | emptying the directory, and never touches the directory itself, which | |
| 425 | on a bind mount logged `unlinkat` and `mkdir` errors every pass (#36). | |
| 426 | The per-platform stat files went with it. x/net bumped (#35). | |
| 427 | ||
| 428 | ### v1.5.5 (!32) | |
| 429 | ||
| 430 | A crawler walking post pages at 70 a minute produced 733 upstream | |
| 431 | timeouts in ten minutes while the address stayed unbanned: each queued | |
| 432 | request still took its interval turn after its client had timed out. | |
| 433 | The throttle now honours the request context, and sheds a request that | |
| 434 | would queue longer than 20 seconds with a 503 and `Retry-After`. The | |
| 435 | instance's `rate-limit` dropped to 20 per minute, burst 10. | |
| 436 | ||
| 437 | ### v1.5.6 (!33, !34) | |
| 438 | ||
| 439 | The real cause of #3: DeviantArt's editor stores descriptions and | |
| 440 | comments as a document tree (`{"version":1,"document":...}`), not | |
| 441 | Draft.js blocks, so every current description rendered empty. A new | |
| 442 | renderer covers the node set seen in 92 live payloads: paragraphs, | |
| 443 | headings, lists, quotes, code, breaks, rules, text marks, emotes (#6), | |
| 444 | embedded artworks, GIF embeds and mentions. Media and post URLs are | |
| 445 | parsed instead of sliced by offset (#1), and old HTML emotes map by | |
| 446 | image name. | |
| 447 | ||
| 448 | ### Closed without code | |
| 449 | ||
| 450 | #2 search filters: DeviantArt's guest search ignores every `order` | |
| 451 | value the site itself uses, and the deviations endpoint redirects | |
| 452 | guests. Nothing to expose. #4 and #5 were covered by the instance | |
| 453 | checker and the Makefile. | |
| 454 | ||
| 455 | ### Lessons | |
| 456 | ||
| 457 | - DeviantArt bans an address on volume, not on whether it is a VPN. | |
| 458 | Pacing, caching and shedding at the instance are what keep it clean; | |
| 459 | rotating exits only buys an hour. | |
| 460 | - A restart empties the in-memory API cache, so a ban right after a | |
| 461 | deploy has nothing stale to serve. Deploy when the instance is quiet. | |
| 462 | - Log driver `none` is a trap. Every incident here was diagnosed from | |
| 463 | lines that driver would have dropped. | |
| 464 | - Fetch real payloads from a host DeviantArt accepts before touching a | |
| 465 | parser; the format had changed under the old one. | |
| 466 | ||