Running Solidtime on Kubernetes with CDK8S (Hetzner + CNPG + ESO)

[jensens]

Hey folks,

We just moved our little agency off Clockify onto self-hosted Solidtime, and figured I'd give something back by dumping how we run it on k8s. Maybe it saves someone an afternoon of head-scratching.

Fat disclaimer first: this is opinionated and very specific to our cluster — k3s on Hetzner, ArgoCD, CloudNativePG, External Secrets, Crossplane. It is not a generic helm install chart. But it's all driven by one config.yaml, and honestly the interesting part isn't the chart — it's the handful of gotchas we hit. Those apply no matter how you deploy.

The stack

• One image, three Deployments via CONTAINER_MODE (http / scheduler / worker) + a Gotenberg sidecar for PDF export
• Postgres via CloudNativePG (2 replicas, barman-cloud backups to S3)
• App files on S3 (Hetzner Object Storage); cache/queue/sessions on the DB, no Redis
• Mailjet for mail, Traefik + cert-manager for ingress/TLS, all secrets via ESO

Gotchas that cost me the most time (the actually useful part):

1. S3 env vars are S3_*, not AWS_*. I reflexively set AWS_ACCESS_KEY_ID & co and the Clockify import blew up with a cryptic AWS-SDK "Missing required client configuration options". config/filesystems.php reads S3_REGION / S3_BUCKET / S3_ENDPOINT / S3_ACCESS_KEY_ID / S3_SECRET_ACCESS_KEY / S3_USE_PATH_STYLE_ENDPOINT (path-style = true for Hetzner).
2. Health probe: hit /login, not / or /health. There's no /health or /up. / 302-redirects to the absolute https:// APP_URL; kubelet then follows that over HTTPS against the plain-HTTP container, which fails with "server gave HTTP response to HTTPS client", so the probe never passes. /login returns a clean 200.
3. The worker needs WORKER_COMMAND. CONTAINER_MODE=worker execs $WORKER_COMMAND; leaving it unset gives you a crashloop with "WORKER_COMMAND is undefined". Set it to php artisan queue:work --tries=3 --max-time=3600.
4. Migrations: AUTO_DB_MIGRATE=true on the scheduler only (single replica) instead of a separate Job — simple and race-free.
5. Passport keys via env. Generate APP_KEY plus a Passport RSA keypair once and inject PASSPORT_PRIVATE_KEY / PASSPORT_PUBLIC_KEY so the pods stay stateless. Never rotate them.
6. Bonus: the built-in Clockify Time Entries importer is genuinely great — placeholder members keyed by email, then invite-placeholder. Just remember the CSV wants US date format and a Billable column.

Full sanitized CDK8S (TypeScript) chart here: https://gist.github.com/jensens/2a24ce088d46a33942f9c066c123c058

Solidtime's been lovely to self-host — thanks for building it. Only thing on the wishlist is an org logo on the PDF export (saw #831, gave it a thumbs up).

Cheers