Five checks, one command, and a number for how far to trust them
A way to test web apps that agents build and nobody reads line by line. It plants bugs in its own code to find out whether a green run means anything.
- Types 15
- Rules 9
- API 2
- Flows 6
- Got through 8
Each check answers a question the others can’t
-
Types
Does the code agree with itself?
Strict TypeScript with no
any, and zod at every boundary: API input, env, forms, storage.bun run typecheck15 of the 40 planted bugsapps/web/server/trpc.ts export const protectedProcedure = t.procedure.meta({ access: 'user' }).use(({ ctx, next }) => { if (!ctx.session) { throw new TRPCError({ code: 'UNAUTHORIZED' }) } return next({ ctx: { ...ctx, session: ctx.session } }) })apps/web/server/demo.ts create: protectedProcedure .input( z.object({ title: z.string().min(1).max(200), content: z.string().min(1).max(2000), }) ) .mutation(async ({ ctx, input }) => { return prisma.post.create({ data: { ...input, authorId: ctx.session.user.id }, }) }),Take the !out of the guard, one of the 40 bugs the planted-bug run tried, and the typecheck names every procedure that assumed a session:$ tsc --noEmit server/demo.ts(35,37): error TS18047: 'ctx.session' is possibly 'null'. server/router.ts(5,51): error TS18047: 'ctx.session' is possibly 'null'.
-
Rules
Is the logic right where a wrong answer is silent?
Pure functions with many cases: money, dates, permissions, parsers, redirects. No database, network or environment.
*.rule.test.ts9 of the 40 planted bugsapps/web/server/access.rule.test.ts import { describe, expect, test } from 'bun:test' import { emailMatches } from './access' describe('emailMatches', () => { const entries = ['ada@example.com', '@acme.io'] test('a whole domain, and only that domain', () => { expect(emailMatches('anyone@acme.io', entries)).toBe(true) expect(emailMatches('x@eng.acme.io', entries)).toBe(false) expect(emailMatches('x@notacme.io', entries)).toBe(false) expect(emailMatches('x@acme.io.evil.com', entries)).toBe(false) }) test('the last @ decides the domain', () => { expect(emailMatches('"a@acme.io"@evil.com', entries)).toBe(false) }) }) -
API
Does every server procedure return, write and refuse what it should, for each kind of caller?
Every tRPC procedure called through the real router on a real Postgres, plus a committed table of who may call what.
*.api.test.ts2 of the 40 planted bugsapps/web/server/demo.api.test.ts import { describe, expect, test } from 'bun:test' import { asAnon, asUser } from '../test/callers' import { prisma } from './prisma' describe('posts.create', () => { test('writes a post owned by the caller and returns it', async () => { const person = await asUser() const created = await person.api.posts.create({ title: 'Hello', content: 'World' }) const row = await prisma.post.findUniqueOrThrow({ where: { id: created.id } }) expect(row).toMatchObject({ title: 'Hello', content: 'World', authorId: person.user.id }) }) test('rejects a signed-out caller and writes nothing', async () => { const before = await prisma.post.count() const anon = await asAnon() await expect(anon.api.posts.create({ title: 'nope', content: 'nope' })).rejects.toMatchObject({ code: 'UNAUTHORIZED', }) expect(await prisma.post.count()).toBe(before) }) })apps/web/server/router.access.md | procedure | kind | signed out | signed in | admin | | --- | --- | --- | --- | --- | | me | query | no | yes | yes | | posts.create | mutation | no | yes | yes | | posts.list | query | yes | yes | yes | -
Flows
Can a person actually do it?
A person’s path through the production build in a real browser, at desktop and phone width, with every screen walked by some flow.
*.flow.spec.ts6 of the 40 planted bugsapps/web/e2e/auth.flow.spec.ts import { expect, test } from './flow' test('auth/sign-in-next · signing in returns to the page and query asked for', async ({ page, world, }) => { // Check: sign-in from /dashboard?tab=posts ends on /dashboard?tab=posts. const person = await world.account('Linus') await page.goto('/dashboard?tab=posts') await page.waitForURL('**/sign-in?next=%2Fdashboard%3Ftab%3Dposts') await page.getByLabel('Email').fill(person.email) await page.getByLabel('Password').fill(person.password) await page.getByRole('button', { name: 'Sign in' }).click() await page.waitForURL((url) => url.pathname === '/dashboard' && url.search === '?tab=posts') await expect(page.getByRole('heading', { level: 1, name: 'Linus' })).toBeVisible() }) -
Live
Is what deployed the thing that was tested, and is it up?
After a deploy: the new revision is running, and the health path and listed paths answer. It warns and never rolls back.
after a deployapps/web/server/health.ts export function healthz(ping: () => Promise<unknown> = pingDatabase): RequestHandler { return async (_req, res) => { try { await ping() res.json({ status: 'ok', uptime: process.uptime(), commit: env.DRYDOCK_COMMIT ?? null }) } catch (e) { log.error(`healthz db check failed: ${formatError(e)}`) res.status(503).json({ status: 'error', error: 'database unreachable' }) } } }apps/web/e2e/health.flow.spec.ts import { expect, test } from './flow' import { FLOW_COMMIT } from './server' test('health/commit · /healthz reports the commit the server was deployed as', async ({ request, }) => { // Check: the body holds `"commit":"<DRYDOCK_COMMIT>"`, the text Drydock's Live check looks for. const res = await request.get('/healthz') expect(res.status()).toBe(200) expect(await res.text()).toContain(`"commit":"${FLOW_COMMIT}"`) })The check's half is a few lines in the app's deploy manifest, written by the deploy tool when the check is turned on. {sha}stands for the commit that was just deployed, so an old version still serving is reported:liveCheck: # after each deploy: report whether the new commit serves (never fails it) paths: - path: "/healthz" contains: "\"commit\":\"{sha}\""
One command runs them on a clean copy of the commit
- It copies the commit, not the working tree. Uncommitted and untracked files are left behind.
- It creates a fresh, migrated database for the run and hands the checks its address.
- It runs each check with an allowlisted environment, so a variable your shell exports can’t make one pass.
- It fails a check that skipped a test, marked one todo, or found none.
- It writes a signed receipt for the commit on a full pass.
$ gate gate sal-starter c9c25a3 report clean copy · 1 database pass setup 2.3s 1 database from the template cache pass types 1.4s pass rules 0.3s 53 tests pass api 3.2s 32 tests pass flows 18.0s 32 tests receipt written · report mode never blocks a push
{
"version": 1,
"mode": "report",
"branches": ["main", "master"],
"copy": { "env": [".env"], "keep": ["node_modules"] },
"setup": ["bun install --frozen-lockfile", "bun run db:generate"],
"databases": [
{
"env": "DATABASE_URL",
"migrate": "bun run db:deploy",
"key": ["apps/web/prisma/schema.prisma", "apps/web/prisma/migrations"]
}
],
"ports": [],
"pass": [],
"checks": [
{ "name": "types", "run": "bun run typecheck" },
{ "name": "rules", "run": "bun run test:rules" },
{ "name": "api", "run": "bun run test:api" },
{ "name": "flows", "run": "bun run test:flows", "timeout": 900 }
],
"probe": {
"include": ["apps/web/server.ts", "apps/web/server/**/*.ts", "apps/web/src/**/*.{ts,tsx}"],
"exclude": [
{ "path": "apps/web/server/logger.ts", "why": "log colours and formatting" },
{ "path": "apps/web/**/*.test.ts", "why": "the tests themselves, not code under test" }
]
}
}One more shows every test and how it last ran
$ gate tests gate tests sal-starter c9c25a3 last run 2026-11-21 16:00 · c9c25a3 · pass · HEAD has a receipt probe 2026-11-21 · 6aa609c · caught 80% · 8 missed types no tests pass 1.4s rules 53 tests 53 pass 0.3s api 32 tests 32 pass 3.2s flows 16 flows at 2 widths 32 pass 18.0s … flows desktop phone ✓ 10 apps/web/e2e/auth.flow.spec.ts caught 11 planted bugs auth/sign-up · an 8-character password signs a new person up onto their dashb… ✓ 1.2s ✓ 1.1s auth/sign-up-short-password · a 7-character password is refused in the form ✓ 0.7s ✓ 0.8s auth/sign-up-next · signing up from a protected page lands back on it ✓ 1.0s ✓ 1.2s auth/protected-redirect · a signed-out visit to the dashboard asks to sign in… ✓ 0.6s ✓ 0.6s auth/sign-in-next · signing in returns to the page and query asked for ✓ 1.4s ✓ 1.1s auth/off-site-next · an off-site ?next= is ignored ✓ 1.2s ✓ 0.9s auth/sign-out · signing out lands home, and Back does not reopen the dashboard ✓ 0.8s ✓ 1.1s auth/dashboard-ssr · a hard load of the dashboard hands React Query a real Da… ✓ 0.8s ✓ 0.7s auth/session-check-retry · one failed session check on a navigation still loa… ✓ 1.9s ✓ 2.0s auth/session-check-error · a session check that keeps failing shows the error… ✓ 2.0s ✓ 2.0s …
gate already kept: each test’s last result, how long it took, and whether it has failed before.--failed for what is not a clean pass, --all for every test by name, --json for an agent.Four rules keep a green run honest
A check has to have been seen failing once
Break the code on purpose, or let the planted-bug run do it, and watch the check go red before trusting its green.
Without it, half of 66 planted bugs passed every check.
The builder doesn’t write its own pass mark
What a flow must show is written down before the code, as one line a person could check by hand. A separate agent turns that line into the browser test.
Without it, the agent that wrote the bug writes the test that agrees with it.
A skipped test is a failed test
No test.skip, no test.todo, no run that finds zero tests, no test that quietly passes when an environment variable is missing.
Without it, one repo had 88 of 438 test files skipping themselves.
Checks run on a clean copy of the commit, with their own database
Never the working tree and never the dev database. A file that was not committed is not in the copy.
Without it, 32 test files in one app were writing to the dev database.
A green run is worth what the planted bugs say it is
Only a new sample measures the checks. Replaying a list you have already fixed goes to 100% and says little.
A missed bug is a real gap. Write the test that catches that exact change, then plant a new sample.
Every kind of repo has a path onto it
A new project starts already on it
- Clone the starter and run
bun run init my-appIt carries the three names, the test databases, the flow fixture, the every-screen check, the access table and agate.json. - Run the kickoff, then
bun run demo:dropThe starter’s own tests fail until both are done. - Run
gatebefore the first feature It should be green. Every later number depends on it. - Build each change check first Written with the change, seen failing, then made to pass.
- List the critical files in
gate.jsonSign-in, permissions, money, anything that writes data. - Plant bugs once there is real code
gate probe --max 40 --seed 1, a test for each miss, then a new seed. - Turn on the live check with the first deploy
- Keep the click-through list from the first flow
An existing app takes four steps, in a fixed order
- Give the tests their own databases A cloned template per test process, one guard on every drop and truncate, no real keys. Done when
gateis green on a clean copy. - Names, the access table, and an API test for every procedure Done when a test names any procedure without one, and names none.
- Flows on every screen, at both widths Done when a full run names any screen no flow walked, and names none.
- Prove it with planted bugs Replay the first list, then plant a new sample and report that number.
An older fork gets it through the starter’s sync
- On a branch, run
bun run syncIt merges the starter’s changes and never commits. - Check
gate.jsonsaysreportIf it doesn’t, or the fork has no web app, runbun scripts/gate.ts --fit. - Run
bun run test:rulesThe names check lists every test file with the name to use. - Run all four checks, commit, run
gate - Plant bugs The fork now stands at Walked. A new sample takes it to Measured.
Something that isn’t a web app keeps the checks that apply
| Types | Rules | API | Flows | Live | |
|---|---|---|---|---|---|
| A web app | Yes | Yes | Every server procedure | Every screen, both widths | Yes |
| A desktop app | Yes | Yes | The bridge between the window and the main process | Only for a web surface | No |
| A mobile app | Yes | Yes | Its server’s procedures, in the server’s repo | Not covered: a real device is a person’s check | No |
| A CLI or a library | Yes | Yes | Each command or exported function, called the way a caller calls it | No screen, no flows | No |
Where a repo stands
- Not on itNo
gate.json - Safe to run
gateis green on a clean copy, with its own databases - NamedThree file names, the access table, an API test for every procedure
- WalkedEvery screen walked by a flow at both widths
- MeasuredA planted-bug run on a new sample is on recordThe starter 80%A production app 63%
What it does not do
- It does not judge taste. Whether a screen feels right is a person’s call.
- It does not stop a push or fail a deploy unless the owner turns that on.
- It does not make green mean “no bugs”. The number is the point: it is known, and it can be moved.
- It does not cover what no check reaches: mobile builds on real devices, third-party dashboards, and email as it lands in an inbox.