Add robots.txt and a per-client rate limit !10

Stacked on !9; retarget to main once that merges.

A token bucket per client address for page, feed and API requests; media, avatars and static files are exempt. Over budget answers 429 with Retry-After and no Cache-Control. Behind a reverse proxy the client is the rightmost X-Forwarded-For entry, trusted only when the connection itself comes from a loopback or private address, so a direct client cannot spoof it. Config block rate-limit with per-minute (60) and burst (20); 0 disables. Idle buckets are pruned.

/robots.txt disallows search, the API, user and group pages, media and ?p= URLs under the base URI, with Crawl-delay: 10.

Seven new tests: burst then refuse, refill rate with a fake clock, pruning, client address resolution, robots.txt content, and the real handler limiting pages but not media.

Closes #10