Skip to content
All posts
3 min read

The site renders with the database down

Every public read on this site has a committed fallback, so a dead database degrades to slightly stale content instead of a blank page.

EngineeringSystems

This week I moved every piece of content on this site, projects, posts, links, out of code and into MySQL on the same host that serves the pages. The first thing I built after the migration was the ability to live without it.

The mechanism is small. The seed content that fills an empty database is also committed in the repo, and every public read path catches database failure and serves the seed instead. The catch in src/features/blog/index.ts is representative: try the posts table, and on any error, log loudly and return the committed posts sorted by date. The page renders either way. The same fallback carries the build: next build runs in an environment with no database credentials at all, and without it the static pages could not even compile.

The interesting bug came two days after the migration. My first version memoized the database initialization promise, which is the obvious move, you do not want every request re-checking whether tables exist. Except a transient failure at boot meant the memo held a rejected promise, and every request for the rest of the process lifetime replayed that rejection. A thirty-second blip became an outage that only a redeploy could end. The fix is a rule I now apply everywhere: never cache a poisoned init. A failed guard resets itself so the next request retries; only success is worth remembering.

Setup is folded into the same philosophy. Content tables bootstrap themselves on first query: the DDL lives next to the connection code, an empty table gets its seed rows inserted once, and there is no migration step to remember on a new machine. A fresh clone with database credentials becomes a working site on the first page load. The database is treated as a cache of the truth the repo can reconstruct, which is an unusual power balance, and for a single-owner content site I think it is the correct one.

There is a second, quieter decision in how the two halves of the site fail. Public pages degrade silently: seed content, a log line, no banner, no apology. The share links got the same hardening this week, surviving transient database blips instead of 500ing, because a link I already sent someone must not depend on my database having a good minute. Admin pages do the opposite and surface the real error, because the only person who will ever see an admin error is me, and I want the stack trace, not a soothing empty state. I think blanket graceful degradation is a mistake. Degrade for guests, fail loud for owners. A fallback that hides a broken database from its own operator is not resilience, it is anesthesia.

The tradeoff is staleness, and it is real. The seed is a snapshot from whenever I last updated it, so during an outage the site can show a project list that is one edit behind. For a personal site, stale beats blank so decisively it is barely a decision. If freshness were the product, a news site, anything with prices, the same design would be wrong.

My test for all of this is not clever: stop the local MariaDB and reload. The site renders like nothing happened while the terminal fills with database-unavailable noise. Both halves of that sentence are the feature. The reader sees the storefront; I see the alarm.

Uptime for the public site now reduces to an arithmetic I can live with: the content is at most one deploy old, no matter what the database is doing.

Remiel Shirazi

CTO & Co-founder | AI, Roblox & Piano