flowingkhaosSign inBrowse the marketplace
New here? flowingkhaos is a build-in-public workshop. This is one part of one of six production failures.What this site is

monitoringFix 01 · part three of five

Investigation — A throwaway test run rewrote our live database

Investigation

12 August. The build loop generated 20260812_040730_payments_ledger and ran payload migrate for real. It landed in the production journal as batch 4, hours before any human reviewed the diff. We found it at land time, when the pre-push migration check found the table already there. Harmless only because the table was empty. First reading: the runner leaked an environment into the gate. Mitigation: query the production migrations table before every land. That finds damage after the fact, which is all such a check can do.

26 August. Migration 0006_safe_champions created and applied from a run on a different feature. Same shape.

6 October. We read the runner’s own source. It has stripped the caller’s working-directory environment since version 0.3.6, on both of our machines’ versions. The runner was not the path.

The real path. No shell profile and no runner env file sets a database URL. The leak was the run itself: an agent or a gate that needed a database found the main checkout’s .env or apps/cms/.env.local, and the migration tool loads those files as it is meant to. The gate never needed the shell’s environment; the checkout carried its own.

Why it was quiet. A real migration connects, applies, and prints a success line. The only difference on screen was a database host name nobody read.

Belief Status Evidence
The runner leaked an environment wrong runner source: env stripping since 0.3.6
A worktree never holds the CMS env file wrong an old gate asserts it is absent, because an agent had copied it in
Env files never override a set variable right bun and Next’s env loader both keep a variable that is already set
The column · weekly · free

The next fix, in your inbox the week it happens.

One written column a week. The email service holds the list.

The marketplacePay once

Skip the build. Start from code that ships

Production-ready code under a one-time licence. Buy it, and start building with the code you bought.

  • Pay once
  • Resell what you build
  • No subscription

The columns, fixes and workshops continue in the community: SEO, AI, and making money with code.

Join on Whop

Luke Sidney writes and builds everything here, alone, in the open. lukesidney.me

Next part · 04 of 05Repair