docs/superpowers/plans/2026-09-10-api-cache.md

v1.5.5

skunky-art/docs/superpowers/plans/2026-09-10-api-cache.md rendered · source · history · blame · raw

776 lines · 20499 bytes

API Response Cache Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Cache DeviantArt API responses in memory so repeat and concurrent page views make one upstream request instead of many.

Architecture: A caching http.RoundTripper sits in front of the existing throttle. It stores 200 responses for /_puppy/ and /groups/ GETs keyed by URL minus csrf_token, bounded by bytes with LRU eviction, with singleflight coalescing of concurrent misses.

Tech Stack: Go 1.25, golang.org/x/sync/singleflight, standard library container/list.

Spec: docs/superpowers/specs/2026-09-10-api-cache-design.md

Global Constraints

Go module is skunkyart; package under test is app.
Lint must stay clean: go run github.com/golangci/golangci-lint/v2/cmd/golangci-lint@v2.13.2 run ./....
Tests run with go test ./app -count=1 -race.
No attribution trailers in commits.
Errors and log lines are plain sentences, lowercase error strings.

Task 1: Shared lifetime parser and the `api-cache` config block

Files:

Modify: app/config.go
Create: app/config_test.go
Modify: config.example.json, SETUP.md

Interfaces:

Produces: func parseLifetime(s string) (time.Duration, error); CFG.APICache of type apiCacheConfig{Enabled bool; MaxSize int64; TTL string}; package var apiCacheTTL time.Duration set by ExecuteConfig.
Step 1: Write the failing tests

package app

import (
	"testing"
	"time"
)

func TestParseLifetimeUnits(t *testing.T) {
	cases := map[string]time.Duration{
		"5i":  5 * time.Minute,
		"2h":  2 * time.Hour,
		"3d":  72 * time.Hour,
		"1w":  7 * 24 * time.Hour,
		"1m":  30 * 24 * time.Hour,
		"1y":  360 * 24 * time.Hour,
		"12i": 12 * time.Minute,
	}
	for in, want := range cases {
		got, err := parseLifetime(in)
		if err != nil || got != want {
			t.Errorf("parseLifetime(%q) = %v, %v; want %v", in, got, err, want)
		}
	}
}

func TestParseLifetimeRejectsBadInput(t *testing.T) {
	for _, in := range []string{"", "5", "5x", "h"} {
		if _, err := parseLifetime(in); err == nil {
			t.Errorf("parseLifetime(%q) accepted, want an error", in)
		}
	}
}

func TestAPICacheDefaults(t *testing.T) {
	if !CFG.APICache.Enabled || CFG.APICache.MaxSize != 64 || CFG.APICache.TTL != "5i" {
		t.Errorf("defaults are %+v, want enabled, 64 MB, 5i", CFG.APICache)
	}
}

Step 2: Run to verify it fails

Run: go test ./app -run 'TestParseLifetime|TestAPICacheDefaults' -count=1 Expected: FAIL, undefined: parseLifetime.

Step 3: Implement

In app/config.go, add "errors" to the imports. Add the type and the field:

type apiCacheConfig struct {
	Enabled bool   `json:"enabled"`
	MaxSize int64  `json:"max-size"`
	TTL     string `json:"ttl"`
}

In config, after Cache:

	APICache      apiCacheConfig `json:"api-cache"`

In the CFG literal, after the Cache: block:

	APICache: apiCacheConfig{
		Enabled: true,
		MaxSize: 64,
		TTL:     "5i",
	},

After var lifetimeParsed int64:

// apiCacheTTL is api-cache.ttl parsed, set by ExecuteConfig.
var apiCacheTTL time.Duration

// parseLifetime reads a duration in the config's unit syntax: a number
// followed by i (minutes), h (hours), d (days), w (weeks), m (30-day
// months) or y (360-day years).
func parseLifetime(s string) (time.Duration, error) {
	if s == "" {
		return 0, errors.New("empty lifetime")
	}
	numstr := regexp.MustCompile("[0-9]+").FindAllString(s, -1)
	if len(numstr) == 0 {
		return 0, errors.New("lifetime has no number: " + s)
	}
	num, _ := strconv.Atoi(numstr[len(numstr)-1])

	day := 24 * time.Hour
	var unit time.Duration
	switch s[len(s)-1:] {
	case "i":
		unit = time.Minute
	case "h":
		unit = time.Hour
	case "d":
		unit = day
	case "w":
		unit = 7 * day
	case "m":
		unit = 30 * day
	case "y":
		unit = 360 * day
	default:
		return 0, errors.New("invalid unit specified: " + s[len(s)-1:])
	}
	return unit * time.Duration(num), nil
}

Replace the if CFG.Cache.Lifetime != "" { ... } block inside ExecuteConfig with:

			if CFG.Cache.Lifetime != "" {
				d, err := parseLifetime(CFG.Cache.Lifetime)
				if err != nil {
					exit("config: cache.lifetime: "+err.Error(), 1)
				}
				lifetimeParsed = d.Milliseconds()
			}

After the theme switch, before static.StaticPath = CFG.StaticPath:

		if CFG.APICache.Enabled {
			d, err := parseLifetime(CFG.APICache.TTL)
			if err != nil {
				exit("config: api-cache.ttl: "+err.Error(), 1)
			}
			apiCacheTTL = d
		}

Remove the now-unused day and duration locals from ExecuteConfig.

In config.example.json, after the cache block:

    "api-cache": {
        "enabled": true,
        "max-size": 64,
        "ttl": "5i"
    },

In SETUP.md, after the cache bullet list, add:

* `api-cache` — In-memory cache of DeviantArt API responses. Every page,
  feed poll and API call that asks DeviantArt the same question within the
  TTL is answered from memory, and concurrent requests for one thing make
  one upstream call. On by default; DeviantArt bans egress IPs that ask too
  often, so leave it on unless you are debugging.
  * `enabled` — boolean, default true
  * `max-size` — megabytes of response bodies to hold, default 64. Least
    recently used entries are dropped past this.
  * `ttl` — how long a response is reused, in the time units above. Default
    `5i`.

Also add d — days to the Time units list at the top of SETUP.md.

Step 4: Run tests

Run: go test ./app -count=1 -race Expected: PASS.

Step 5: Commit

git add app/config.go app/config_test.go config.example.json SETUP.md
git commit -m "Add the api-cache config block and a shared lifetime parser

Ref #8"

Task 2: The cache store and transport

Files:

Create: app/apicache.go, app/apicache_test.go
Modify: go.mod, go.sum

Interfaces:

Consumes: nothing from Task 1 (the store takes its bound and TTL as arguments).
Produces: func newAPICache(maxBytes int64, ttl time.Duration) *apiCache; func (c *apiCache) transport(base http.RoundTripper) http.RoundTripper; func (c *apiCache) stats() (hits, misses int64, entries int, held int64); func cacheable(req *http.Request) bool; func cacheKey(req *http.Request) string.
Step 1: Add the dependency

Run: go get golang.org/x/sync@latest && go mod tidy

Step 2: Write the failing tests

package app

import (
	"io"
	"net/http"
	"strings"
	"sync"
	"testing"
	"time"
)

// fakeRT is the upstream: it counts calls and returns a scripted response.
type fakeRT struct {
	mu     sync.Mutex
	calls  int
	status int
	body   string
	delay  time.Duration
}

func (f *fakeRT) RoundTrip(r *http.Request) (*http.Response, error) {
	f.mu.Lock()
	f.calls++
	f.mu.Unlock()
	time.Sleep(f.delay)
	return &http.Response{
		StatusCode: f.status,
		Header:     http.Header{"Content-Type": {"application/json"}},
		Body:       io.NopCloser(strings.NewReader(f.body)),
		Request:    r,
	}, nil
}

func (f *fakeRT) count() int {
	f.mu.Lock()
	defer f.mu.Unlock()
	return f.calls
}

const puppyURL = "https://www.deviantart.com/_puppy/dabrowse/search/all?q=fox&csrf_token=abc"

func get(t *testing.T, rt http.RoundTripper, url string) (int, string) {
	t.Helper()
	req, err := http.NewRequest(http.MethodGet, url, nil) //nolint:noctx // test request
	if err != nil {
		t.Fatal(err)
	}
	resp, err := rt.RoundTrip(req)
	if err != nil {
		t.Fatal(err)
	}
	defer func() { _ = resp.Body.Close() }()
	body, _ := io.ReadAll(resp.Body)
	return resp.StatusCode, string(body)
}

func TestSecondRequestIsServedFromCache(t *testing.T) {
	up := &fakeRT{status: 200, body: `{"a":1}`}
	rt := newAPICache(1<<20, time.Minute).transport(up)

	get(t, rt, puppyURL)
	status, body := get(t, rt, puppyURL)

	if up.count() != 1 {
		t.Errorf("upstream called %d times, want 1", up.count())
	}
	if status != 200 || body != `{"a":1}` {
		t.Errorf("cached response is %d %q", status, body)
	}
}

func TestExpiredEntryIsRefetched(t *testing.T) {
	up := &fakeRT{status: 200, body: `{}`}
	c := newAPICache(1<<20, time.Minute)
	now := time.Now()
	c.now = func() time.Time { return now }
	rt := c.transport(up)

	get(t, rt, puppyURL)
	now = now.Add(2 * time.Minute)
	get(t, rt, puppyURL)

	if up.count() != 2 {
		t.Errorf("upstream called %d times, want 2 after expiry", up.count())
	}
}

func TestNon200IsNotStored(t *testing.T) {
	up := &fakeRT{status: 403, body: "blocked"}
	rt := newAPICache(1<<20, time.Minute).transport(up)

	status, body := get(t, rt, puppyURL)
	get(t, rt, puppyURL)

	if status != 403 || body != "blocked" {
		t.Errorf("first response is %d %q, want the upstream 403 passed through", status, body)
	}
	if up.count() != 2 {
		t.Errorf("upstream called %d times, want 2: a 403 must not be cached", up.count())
	}
}

func TestBypassesSessionAndOtherHosts(t *testing.T) {
	up := &fakeRT{status: 200, body: "x"}
	rt := newAPICache(1<<20, time.Minute).transport(up)

	for _, url := range []string{
		"https://www.deviantart.com/_puppy",
		"https://www.deviantart.com",
		"https://a.deviantart.net/avatars-big/a/alice.png",
	} {
		get(t, rt, url)
		get(t, rt, url)
	}
	if up.count() != 6 {
		t.Errorf("upstream called %d times, want 6: none of these URLs may be cached", up.count())
	}
}

func TestKeyIgnoresCSRFToken(t *testing.T) {
	up := &fakeRT{status: 200, body: "x"}
	rt := newAPICache(1<<20, time.Minute).transport(up)

	get(t, rt, puppyURL)
	get(t, rt, strings.Replace(puppyURL, "csrf_token=abc", "csrf_token=def", 1))

	if up.count() != 1 {
		t.Errorf("upstream called %d times, want 1: a token refresh must not miss", up.count())
	}
}

func TestByteBoundEvictsLeastRecentlyUsed(t *testing.T) {
	up := &fakeRT{status: 200, body: strings.Repeat("x", 100)}
	rt := newAPICache(250, time.Minute).transport(up)
	a := "https://www.deviantart.com/_puppy/a?p=1"
	b := "https://www.deviantart.com/_puppy/b?p=1"
	c := "https://www.deviantart.com/_puppy/c?p=1"

	get(t, rt, a)
	get(t, rt, b)
	get(t, rt, a) // a is now more recent than b
	get(t, rt, c) // 300 bytes would exceed 250: b goes
	get(t, rt, a)
	get(t, rt, c)
	get(t, rt, b)

	if up.count() != 4 {
		t.Errorf("upstream called %d times, want 4: only b should have been evicted", up.count())
	}
}

func TestConcurrentMissesMakeOneUpstreamCall(t *testing.T) {
	up := &fakeRT{status: 200, body: "x", delay: 50 * time.Millisecond}
	rt := newAPICache(1<<20, time.Minute).transport(up)

	var wg sync.WaitGroup
	for range 20 {
		wg.Go(func() { get(t, rt, puppyURL) })
	}
	wg.Wait()

	if up.count() != 1 {
		t.Errorf("upstream called %d times, want 1 for a burst on one key", up.count())
	}
}

func TestStatsCountHitsAndMisses(t *testing.T) {
	up := &fakeRT{status: 200, body: "abc"}
	c := newAPICache(1<<20, time.Minute)
	rt := c.transport(up)

	get(t, rt, puppyURL)
	get(t, rt, puppyURL)
	get(t, rt, puppyURL)

	hits, misses, entries, held := c.stats()
	if hits != 2 || misses != 1 || entries != 1 || held != 3 {
		t.Errorf("stats = %d hits, %d misses, %d entries, %d bytes; want 2, 1, 1, 3", hits, misses, entries, held)
	}
}

Note: wg.Go needs Go 1.25, which go.mod already requires.

Step 3: Run to verify it fails

Step 4: Implement app/apicache.go

package app

import (
	"bytes"
	"container/list"
	"io"
	"net/http"
	"strings"
	"sync"
	"time"

	"golang.org/x/sync/singleflight"
)

// apiCache holds DeviantArt API responses so that repeat and concurrent
// requests for one URL cost one upstream call. It sits in front of the
// throttle: a hit never touches DeviantArt or the throttle's budget.
//
// The store is bounded by body bytes with least-recently-used eviction. It
// is safe for concurrent use.
type apiCache struct {
	maxBytes int64
	ttl      time.Duration
	now      func() time.Time

	mu      sync.Mutex
	entries map[string]*cacheEntry
	lru     *list.List // front is most recently used
	held    int64
	hits    int64
	misses  int64

	flight singleflight.Group
}

// cacheEntry is one buffered response. header is a clone of the upstream
// header; body is the whole body, read once.
type cacheEntry struct {
	key     string
	status  int
	header  http.Header
	body    []byte
	expires time.Time
	elem    *list.Element
}

func newAPICache(maxBytes int64, ttl time.Duration) *apiCache {
	return &apiCache{
		maxBytes: maxBytes,
		ttl:      ttl,
		now:      time.Now,
		entries:  map[string]*cacheEntry{},
		lru:      list.New(),
	}
}

// cacheable reports whether a request is one the cache handles: a GET to
// DeviantArt's API or its group search page. The session bootstrap (/_puppy
// with no path), the homepage, avatars and media all pass through.
func cacheable(req *http.Request) bool {
	if req.Method != http.MethodGet || req.URL.Host != "www.deviantart.com" {
		return false
	}
	p := req.URL.Path
	return (strings.HasPrefix(p, "/_puppy/") && len(p) > len("/_puppy/")) ||
		strings.HasPrefix(p, "/groups/")
}

// cacheKey is the URL without csrf_token, which changes every twelve hours
// and would otherwise empty the cache on each refresh.
func cacheKey(req *http.Request) string {
	u := *req.URL
	q := u.Query()
	q.Del("csrf_token")
	u.RawQuery = q.Encode()
	return u.String()
}

// transport returns a RoundTripper that answers from this cache and sends
// misses to base. Several transports may share one cache.
func (c *apiCache) transport(base http.RoundTripper) http.RoundTripper {
	return &cachedTransport{cache: c, base: base}
}

type cachedTransport struct {
	cache *apiCache
	base  http.RoundTripper
}

// RoundTrip serves a hit from memory. A miss is fetched once per key however
// many callers are waiting, buffered, stored if it is a 200, and handed to
// every waiter as its own response.
func (t *cachedTransport) RoundTrip(req *http.Request) (*http.Response, error) {
	if !cacheable(req) {
		return t.base.RoundTrip(req)
	}
	key := cacheKey(req)
	if e := t.cache.get(key); e != nil {
		return e.response(req), nil
	}

	v, err, _ := t.cache.flight.Do(key, func() (any, error) {
		resp, err := t.base.RoundTrip(req)
		if err != nil {
			return nil, err
		}
		defer func() { _ = resp.Body.Close() }()
		body, err := io.ReadAll(resp.Body)
		if err != nil {
			return nil, err
		}
		e := &cacheEntry{key: key, status: resp.StatusCode, header: resp.Header.Clone(), body: body}
		if e.status == http.StatusOK {
			t.cache.put(e)
		}
		return e, nil
	})
	if err != nil {
		return nil, err
	}
	e, ok := v.(*cacheEntry)
	if !ok {
		return nil, io.ErrUnexpectedEOF
	}
	return e.response(req), nil
}

// response builds a fresh http.Response over the buffered body, so each
// caller can read and close its own.
func (e *cacheEntry) response(req *http.Request) *http.Response {
	return &http.Response{
		Status:        http.StatusText(e.status),
		StatusCode:    e.status,
		Proto:         "HTTP/1.1",
		ProtoMajor:    1,
		ProtoMinor:    1,
		Header:        e.header.Clone(),
		Body:          io.NopCloser(bytes.NewReader(e.body)),
		ContentLength: int64(len(e.body)),
		Request:       req,
	}
}

// get returns the live entry for key, marking it most recently used, or nil.
// An expired entry is dropped on the way out.
func (c *apiCache) get(key string) *cacheEntry {
	c.mu.Lock()
	defer c.mu.Unlock()

	e := c.entries[key]
	if e == nil {
		c.misses++
		return nil
	}
	if !c.now().Before(e.expires) {
		c.remove(e)
		c.misses++
		return nil
	}
	c.lru.MoveToFront(e.elem)
	c.hits++
	return e
}

// put stores e, evicting from the least recently used end until it fits. A
// body larger than the whole bound is not stored.
func (c *apiCache) put(e *cacheEntry) {
	size := int64(len(e.body))
	if size > c.maxBytes {
		return
	}

	c.mu.Lock()
	defer c.mu.Unlock()

	if old := c.entries[e.key]; old != nil {
		c.remove(old)
	}
	for c.held+size > c.maxBytes {
		back, ok := c.lru.Back().Value.(*cacheEntry)
		if !ok {
			break
		}
		c.remove(back)
	}
	e.expires = c.now().Add(c.ttl)
	e.elem = c.lru.PushFront(e)
	c.entries[e.key] = e
	c.held += size
}

// remove drops e. The caller holds mu.
func (c *apiCache) remove(e *cacheEntry) {
	c.lru.Remove(e.elem)
	delete(c.entries, e.key)
	c.held -= int64(len(e.body))
}

// stats reports the counters for the hourly log line.
func (c *apiCache) stats() (hits, misses int64, entries int, held int64) {
	c.mu.Lock()
	defer c.mu.Unlock()
	return c.hits, c.misses, len(c.entries), c.held
}

Step 5: Run tests

Run: go test ./app -count=1 -race Expected: PASS.

Step 6: Lint

Run: go run github.com/golangci/golangci-lint/v2/cmd/golangci-lint@v2.13.2 run ./... Expected: 0 issues.

Step 7: Commit

git add app/apicache.go app/apicache_test.go go.mod go.sum
git commit -m "Add an in-memory cache for DeviantArt API responses

Ref #8"

Task 3: Install the cache in the transport chain

Files:

Modify: app/httpclient.go
Modify: app/apicache_test.go (one more test)
Modify: app/httpclient_test.go if it constructs the chain directly (read it first)

Interfaces:

Consumes: newAPICache, (*apiCache).transport, (*apiCache).stats, CFG.APICache, apiCacheTTL, throttled, daThrottle.
Produces: package var daCache *apiCache (nil when disabled); func chain(base http.RoundTripper) http.RoundTripper.
Step 1: Write the failing test (append to app/apicache_test.go)

// TestHitDoesNotConsumeAThrottleSlot pins the chain order: the cache sits in
// front of the throttle, so a hit returns even when every throttle slot is
// held. With the order reversed this test hangs and times out.
func TestHitDoesNotConsumeAThrottleSlot(t *testing.T) {
	up := &fakeRT{status: 200, body: "x"}
	th := &daThrottle{base: up, sem: make(chan struct{}, 1)}
	rt := newAPICache(1<<20, time.Minute).transport(th)

	get(t, rt, puppyURL) // populate through the throttle

	th.sem <- struct{}{} // hold the only slot
	done := make(chan struct{})
	go func() {
		get(t, rt, puppyURL)
		close(done)
	}()
	select {
	case <-done:
	case <-time.After(2 * time.Second):
		t.Fatal("a cache hit waited on the throttle")
	}
}

Step 2: Run to verify it fails

Run: go test ./app -run TestHitDoesNotConsumeAThrottleSlot -count=1 Expected: PASS already, since the test builds the chain by hand. That is fine: it documents the required order, and Step 3 makes the production chain match it.

Step 3: Implement

In app/httpclient.go, after var baseTransport *http.Transport:

// daCache is the API response cache shared by every transport, or nil when
// api-cache.enabled is false.
var daCache *apiCache

// chain wraps base with the throttle and, when enabled, the cache in front
// of it, so a hit never spends a throttle slot.
func chain(base http.RoundTripper) http.RoundTripper {
	rt := throttled(base)
	if daCache != nil {
		return daCache.transport(rt)
	}
	return rt
}

// logCacheStatsForever prints one line an hour so an operator can see the
// cache working without an endpoint. Run it in its own goroutine.
func logCacheStatsForever(c *apiCache) {
	for {
		time.Sleep(time.Hour)
		hits, misses, entries, held := c.stats()
		println("api cache:", hits, "hits,", misses, "misses,", entries, "entries,", held>>20, "MB held")
	}
}

Change InstallDAThrottle to:

func InstallDAThrottle() {
	baseTransport = tunedTransport()
	if CFG.APICache.Enabled {
		daCache = newAPICache(CFG.APICache.MaxSize<<20, apiCacheTTL)
		go logCacheStatsForever(daCache)
	}
	http.DefaultTransport = chain(baseTransport)
}

Update its doc comment to mention the cache. In ProxiedTransport, replace return throttled(base) with return chain(base).

Step 4: Run tests and lint

Run: go test ./... -count=1 -race && go run github.com/golangci/golangci-lint/v2/cmd/golangci-lint@v2.13.2 run ./... Expected: PASS, 0 issues.

Step 5: Commit

git add app/httpclient.go app/apicache_test.go
git commit -m "Serve DeviantArt API responses from the cache

The cache sits in front of the throttle, so a hit costs neither an
upstream request nor a throttle slot. On by default; api-cache.enabled
turns it off.

Closes #8"

docs/superpowers/plans/2026-09-10-api-cache.md

API Response Cache Implementation Plan

Global Constraints

Task 1: Shared lifetime parser and the api-cache config block

Task 2: The cache store and transport

Task 3: Install the cache in the transport chain

Task 1: Shared lifetime parser and the `api-cache` config block