webweekly

Weekly screenshots of a fixed set of real-world sites, rendered with the latest gosub nightly, published to https://webweekly.gosub.io.

Every Monday a GitHub Action downloads the newest nightly build from gosub-io/gosub-engine, renders every URL in sites.txt, regenerates the browsable portal, and rsyncs the result to the web server. The point is a visual regression record: you can walk a single site forward through the weeks and watch the engine's rendering improve.

Layout on the server

WEBWEEKLY_PATH is the document root:

/                                  → redirect into portal/
portal/index.html                  → browse by week, or by site
portal/sites/<slug>.html           → one site's timeline across every week
2026-W22/ … 2026-W31/              → per-week PNGs + that week's report

How the archive survives an ephemeral runner

generate_portal.py builds its index by globbing <week>/*.png across every week directory on disk — but a CI runner starts empty, and the archive is hundreds of megabytes and growing.

The server is the archive of record. Before generating, the workflow asks it for a recursive file listing (rsync --list-only, which transfers no file data) and recreates every historical PNG locally as a zero-byte stub. The generator only ever reads filenames (Path.stem), never image bytes, so stubs are enough to make every week and site link correctly.

Nothing then uploads those stubs back:

• the week directory is synced additively (no --delete) with rsync --min-size=1, which skips every zero-byte file — a stub can never overwrite a real screenshot;
• portal/ is fully regenerated each run, so it is mirrored with --delete, which prunes pages for sites dropped from sites.txt.

A consequence worth knowing: because the week sync is additive, re-running a week that already has screenshots keeps whatever the server holds for any site that fails on the re-run. capture.sh mirrors that locally — a failed capture with a usable file already on disk is reported as a success rather than downgraded to a failure card.

One-time setup

1. Push this repo to GitHub

The week directories and portal/ are gitignored — they belong on the server, not in git history.

2. Create a deploy key

On the machine that already has access to the server:

ssh-keygen -t ed25519 -f ~/.ssh/webweekly_deploy -C 'webweekly github action' -N ''
ssh-copy-id -i ~/.ssh/webweekly_deploy.pub <user>@webweekly.gosub.io

Consider restricting the key in the server's ~/.ssh/authorized_keys so it can only rsync into the docroot:

command="rrsync /var/www/webweekly",restrict ssh-ed25519 AAAA... webweekly github action

rrsync permits --list-only, so the history step still works under this restriction.

3. Configure the repository

Settings → Secrets and variables → Actions

Kind	Name	Example
Variable	WEBWEEKLY_HOST	webweekly.gosub.io
Variable	WEBWEEKLY_USER	deploy
Variable	WEBWEEKLY_PATH	/var/www/webweekly
Variable	WEBWEEKLY_PORT	22 (optional, defaults to 22)
Secret	WEBWEEKLY_SSH_KEY	contents of webweekly_deploy (the private key)
Secret	WEBWEEKLY_KNOWN_HOSTS	output of ssh-keyscan webweekly.gosub.io

WEBWEEKLY_KNOWN_HOSTS is technically optional — without it the workflow falls back to ssh-keyscan at run time and warns, which trusts whatever answers on first contact. Set it.

4. Seed the server with the existing archive

The weeks currently sitting on your laptop are the only copy. Push them up once:

REMOTE_USER=deploy REMOTE_HOST=webweekly.gosub.io REMOTE_PATH=/var/www/webweekly \
  ./scripts/bootstrap-remote.sh --dry-run     # inspect first
REMOTE_USER=deploy REMOTE_HOST=webweekly.gosub.io REMOTE_PATH=/var/www/webweekly \
  ./scripts/bootstrap-remote.sh

5. Test before trusting the schedule

Run the workflow manually with dry_run = true. It captures everything and generates the portal, but touches neither the server nor its SSH config, and uploads the result as a build artifact you can download and open locally.

Running the workflow

• Scheduled: Mondays 06:00 UTC, comfortably after the engine's 02:00 UTC nightly build.
• Manual: Actions → Weekly capture → Run workflow, with two inputs:
  • week_label — capture into a specific week (e.g. 2026-W31) instead of the current ISO week.
  • dry_run — capture and generate, but do not deploy.

The job summary lists the engine build that produced the run and a per-site ok/failed table. The job fails only if every site fails; partial failures are a warning, since an unreachable site shouldn't block publishing the rest.

Running it locally

./scripts/fetch-nightly.sh                              # → .nightly/gosub-screenshot
GOSUB_SCREENSHOT_BIN=$PWD/.nightly/gosub-screenshot ./capture.sh
python3 ./generate_portal.py

capture.sh honours WEEK_LABEL, VIEWPORT_WIDTH and GOSUB_SCREENSHOT_BIN, and takes an output directory as $1.

Notes

• gosub-screenshot in the nightly tarball is genuinely headless — its only shared-library needs are fontconfig/freetype plus image and compression libs. No GTK, X11, Wayland or Xvfb. (The DISPLAY fallback still in capture.sh is a leftover from an older GTK-backed build and is harmless.)
• The workflow pins runs-on: ubuntu-24.04 to match the environment the nightly is built in. A newer runner image can ship a glibc the prebuilt binary was not linked against.
• Fonts are installed explicitly rather than relying on the runner image's defaults, so renders don't drift week to week when GitHub changes that image.
• Week labels use date +%G-W%V (ISO week-numbering year), not %Y, so the last days of December don't get filed under the wrong year's week 52/53.
• Adding a site is just a line in sites.txt (url|label). It appears in the portal from the next capture onward, with earlier weeks shown as "not captured this week".