docs/superpowers/specs/2026-09-10-api-cache-design.md

  1# API response cache
  2
  3Issue #8. Roadmap item 1.1.
  4
  5## Problem
  6
  7Nothing from DeviantArt's JSON API is cached. Every page view re-fetches
  8its JSON, concurrent requests for one page each go upstream, and feed
  9readers and `/api/random` add calls on a schedule. DeviantArt fronts the
 10API with a WAF that bans egress IPs, so the number of upstream requests,
 11not their pacing, is what gets an instance blocked. The throttle in
 12`app/httpclient.go` spaces requests out; it does not reduce them.
 13
 14## Placement
 15
 16A caching `http.RoundTripper` in `app/apicache.go`. `InstallDAThrottle`
 17builds the chain
 18
 19    devianter -> cache -> throttle -> base transport
 20
 21so a hit is answered before the throttle's rate limiter and semaphore are
 22consulted, and a miss goes through them as today. `ProxiedTransport`
 23builds the same chain over its proxied base, sharing the one cache
 24instance.
 25
 26## Scope
 27
 28Cached: GET requests to host `www.deviantart.com` whose path starts with
 29`/_puppy/` (with something after it) or `/groups/`, when the response
 30status is 200.
 31
 32Passed through, never stored: every other method, host or path. That
 33includes the session bootstrap (`/_puppy` bare), the homepage CSRF scrape,
 34avatars and emotes on `a.deviantart.net` and `e.deviantart.net` (issue #9),
 35and wixmp media. Non-200 responses and transport errors are returned
 36unchanged and not stored, so a WAF block is not remembered.
 37
 38## Key
 39
 40The request URL with the `csrf_token` query parameter removed, otherwise
 41verbatim. The token changes every twelve hours and would otherwise empty
 42the cache on each refresh. The guest cookie is the same for every request
 43and is not part of the key.
 44
 45## Entry
 46
 47Status, `Content-Type`, body bytes, expiry time. A hit returns a new
 48`*http.Response` with those headers, `ContentLength` set, and the body as
 49a `bytes.Reader`. devianter reads the body and closes it as with a live
 50response.
 51
 52## Coalescing
 53
 54`golang.org/x/sync/singleflight` keyed the same as the cache. Concurrent
 55misses for one key make one upstream request; every waiter receives the
 56same stored entry. A miss whose upstream result is not cacheable is still
 57shared with the waiters of that flight, then not stored.
 58
 59## Bounds
 60
 61Bounded by total body bytes. Least-recently-used eviction over a map plus
 62`container/list`, one mutex. A lookup that finds an expired entry removes
 63it and reports a miss. An insert that would exceed the bound evicts from
 64the least recently used end until it fits. A body larger than the bound is
 65served but not stored. No background goroutine.
 66
 67## Configuration
 68
 69New block in `config.json`, beside `cache`:
 70
 71    "api-cache": {
 72        "enabled": true,
 73        "max-size": 64,
 74        "ttl": "5i"
 75    }
 76
 77`enabled` defaults to true. `max-size` is megabytes, default 64. `ttl`
 78uses the same unit syntax as `cache.lifetime` (`i` minutes, `h` hours,
 79`d` days, `w` weeks, `m` months, `y` years), default five minutes. The
 80lifetime parser in `config.go` becomes a function both blocks call; an
 81unparseable value exits at startup with the same message as today.
 82
 83One TTL for every endpoint. Per-endpoint values are not in scope.
 84
 85## Observability
 86
 87Every hour, one line on stdout: hits, misses, entries, bytes held. Nothing
 88else. No endpoint.
 89
 90## Errors
 91
 92Upstream errors and non-200 statuses pass through unchanged. A body read
 93error is returned as an error to the caller and nothing is stored.
 94
 95## Testing
 96
 97`app/apicache_test.go`, with a fake base `RoundTripper` that counts calls
 98and returns scripted responses:
 99
100- second request for one key makes no upstream call
101- a request after expiry refetches
102- a non-200 response is not stored; the next request refetches
103- `/_puppy` bare, the homepage, and a `.net` host bypass the cache
104- two URLs differing only in `csrf_token` share one entry
105- inserting past `max-size` evicts the least recently used entry
106- a burst of concurrent misses for one key produces one upstream call
107- a hit through the full `cached(throttled(base))` chain does not consume
108  a throttle slot: hold the semaphore full, make the hit, observe it
109  return
110
111## Files
112
113- new `app/apicache.go`, `app/apicache_test.go`
114- `app/httpclient.go`: build the chain
115- `app/config.go`: the block, defaults, shared lifetime parser
116- `config.example.json`, `SETUP.md`
117- `go.mod`, `go.sum`: `golang.org/x/sync`