design: benchmark the redesign against rival forges and test newcomer comprehension #226

Review item B08, out of scope for #218. With v1.23.0 deployed, compare representative tasks and visual states (landing, repository overview, a merge request, settings) against Primer/GitHub, SourceHut and one Forgejo instance, and run a newcomer walk: does a first-time visitor understand what gitbay is and complete a first useful action (register, clone, open an issue) without CLI knowledge? Judge by first-action success, clarity and effort, not feature count.

Benchmark, 2026-09-19, gitbay.org at v1.26.0 against github.com/krazywarez/gitbay (the mirror of this repository, so the overview is like for like), codeberg.org (Forgejo: landing, forgejo/forgejo, its pull request list) and sr.ht (landing, ~sircmpwn/legacy.sr.ht). Desktop, 1280 by 900, dark.

Repository overview. GitHub and Forgejo place the About column (description, topics, licence, contributors, languages) beside the file table; gitbay renders the same facts after the README, which on krz/gitbay is a long scroll down. Identity and health are the first thing a visitor wants and the rivals answer it without scrolling. Filed as a follow-up.

Landing. Codeberg leads with a proposition, three pillars and a visible register button. gitbay's copy is as clear, but the picture is tall enough at 1280 by 900 that the facets and the two routes fall below the fold; a newcomer sees the quickstart and a screenshot before any call to action. Filed as a follow-up.

Merge request list. Forgejo's rows carry labels, check state and comment counts; gitbay's carry title and metadata. A reviewer scanning for what needs attention has less to go on. Filed as a follow-up.

SourceHut. Pages are lists of links with no orientation beyond tabs; gitbay is ahead on polish and structure, and matches its no-JavaScript delivery.

Register and sign in. The rivals ask for email and password; gitbay asks for a public key, and the hint, example and wiki link on /register cover the difference. No gap found.

Rail. The icon-only rail (v1.25.0) trades discoverability for space; every glyph has a tooltip and accessible name, but a first visit shows no words. Worth watching in the newcomer test rather than changing now.

What this does not cover: a real newcomer walking the site. That needs a person who has not seen gitbay; the route to test is landing, explore, a repository, register, and a first clone or issue. Leaving this issue open for that half.

Newcomer walk, 2026-09-19, gitbay.org at v1.28.0, anonymous, reading each page as someone who has never used a terminal-first forge.

Landing. The proposition, the two routes and the sign-in line are readable on the first screen. A newcomer clicks Explore or Create an account without needing the quickstart. No stall.

Explore. A list with descriptions, topics and licences; the filter and Search button are obvious. No stall.

Repository. Identity in the header, the file table, then the README; the About column names the licence and build state; clone commands are two labelled blocks. A newcomer without git knowledge does not know what to do with them, and the page does not say; that is expected for a forge and the rivals do not solve it either.

Register. Three fields; the key hint says paste the file's contents, gives the example, and links to a page that makes a key in two commands. The linked page told the reader to run =ssh git@ whoami= with a literal placeholder. Stall one, fixed in the MR below.

After registering. The page says a code was mailed and shows an SSH command to activate; the mail says the same. A pending account can in fact sign in with an emailed link and paste the code under Settings, but neither the page nor the mail says so, so a browser-only newcomer believes the terminal is required. Stall two, fixed in the MR below.

First useful action. The Quickstart wiki page opens with =go install=, which assumes a Go toolchain; the Homebrew tap and the browser route (+ New repository, clone, push) are not mentioned. Stall three, fixed in the MR below.

Rail. Icon-only; every glyph has a tooltip and accessible name, but a first visit shows no words. Watch for this in the human test.

Test script for a real newcomer, ten minutes, no help given. Note where they pause, what they say, and whether they finish each step.

1. Open gitbay.org. Ask: what is this, and what would you do first?
2. Find a repository that interests you and say what it is for.
3. Create an account. Making an SSH key counts as part of the step; note whether the linked page was enough.
4. Verify the account from the mail without using a terminal.
5. Create a repository from the browser and find the clone command.
6. Sign out and sign back in. Score each step done, done with a pause, or abandoned; anything abandoned becomes an issue here.