# Phase A — TDD design brief for the onboarding system > **For Junie in Sonnet mode.** This is the design phase. You produce a > design doc (`docs/plans/onboarding-tdd.md`) plus one hard test fixture. > Phase B (Codex) consumes the design and writes the bulk of the test bodies. > **Do not write production-quality test files in Phase A** — write the spec > they'll be written from. --- ## 1. Why we're slowing down Five commits landed in rapid succession over the last sitting, none with automated test coverage: | Commit | Surface | |---|---| | [`f805404`](#) | `oauth2-proxy --skip-auth-route` for Supabase API surfaces (unifies `db.0.knoe.dev` for Studio + SDK) | | [`79a3f76`](#) | `GRANT anon, authenticated, service_role TO supabase_storage_admin`; oauth2-proxy `--pass/set-authorization-header` removed; `/support` 302→mailto stop-gap | | [`8c2956b`](#) | `pg_stat_statements` moved from `public` to `extensions` schema (durable across all 4 deploy modes) | | [`317b20b`](#) | TCP LB `pg.0.knoe.dev` (34.106.156.196), CNPG cert SAN, `knoe_developer` role, `chrisfu` + `ron` engineer roles, pg_hba tightened | | [`313ee88`](#) | Reusable engineer onboarding: `etc/onboard_engineer.sh`, knoe-onboard nginx pod, `/onboard.html` Kong route, membership-based pg_hba (`+knoe_developer`), committed CA cert | About to onboard the second human engineer (`ron`) against this stack. Standing decision was: **don't add a user, even of one, before we have tests covering the surfaces a user touches**. This brief is that decision being acted on. --- ## 2. What you produce in Phase A Three artifacts, in order: 1. **The f-string fix at `knoe/core/ops/cloudnative_pg.py:1372`** committed first. 30 test-collection errors block the Phase 1 pipeline; until that lands, no test we write has anywhere to run. See top of [`docs/TODO.md`](../TODO.md). Trivial: extract `field.replace('.', '\\.')` to a variable before the f-string. **One commit, message starts `fix(test):`**. 2. **`docs/plans/onboarding-tdd.md`** — the design doc. This is the main deliverable. It must be detailed enough that Codex (Sonnet's successor in Phase B, no shared memory) can write every test body from it without needing to make architectural calls. Sections required: - Test taxonomy (Tier 1 / Tier 2 / Tier 3) with one-line acceptance per test — not bash code, but a precise spec of what's being asserted. - Fixture architecture decision (k3d-per-run vs. shared persistent vs. hybrid) with the tradeoff analysis recorded. - Browser-test framework decision (Playwright vs. Puppeteer vs. ...) with why. - Directory layout under `tests/` and how it slots into the existing pytest structure (read [`tests/test_render_supabase_hostname.py`](../../tests/test_render_supabase_hostname.py) and [`tests/test_repair_update_and_supabase_flags.py`](../../tests/test_repair_update_and_supabase_flags.py) for the conventions to match). - CI integration — these tests slot into Phase 1 of [`docs/pipeline-phases.md`](../pipeline-phases.md). Document the new run config(s), and whether the integration tier needs its own marker (e.g. `pytest -m integration`) so unit tests stay fast for local dev. - Definition of done per tier — what passes count as the boundary covered vs. work-in-progress. 3. **The first hard fixture, working.** Specifically: a deterministic way to simulate "external client connecting to the cluster" inside CI. This is the single hardest design call in Phase A; everything in Tier 1 hangs off it. Get it functional and verified — even if it's just a 30-line `conftest.py` helper — so Codex can rely on it from day one. **One commit, message starts `test(infra):`**. You do **not** write test bodies in Phase A. You write the spec and the hardest fixture. Resist the temptation to land a few "easy" tests — Codex gets a much cleaner working environment if Phase A's output is purely design + fixture. --- ## 3. Test taxonomy (the spec to flesh out) This is what the conversation that produced this brief settled on. Use it as the skeleton for `docs/plans/onboarding-tdd.md` Section "Test taxonomy", but you're expected to refine it: add edge cases, kill duplicates, surface gaps. ### Tier 1 — Security boundaries (must-have before more onboards) The wall protecting the database from the public internet. **No test = no proof.** Hard-gating on the team's deploy pipeline. 1. **pg_hba rejection matrix.** For each axis cross-product: - **Source**: external IP, internal cluster IP (10.0.0.0/8 simulated) - **Encryption**: TLS, plaintext - **Role**: `chrisfu`, `ron`, a fresh-onboarded `knoe_developer` member, `postgres`, `supabase_admin`, `anon`, a non-member of `knoe_developer` with valid LOGIN + password - The matrix has 36 cells minimum. Acceptance: each cell asserts the expected `connect / reject / auth-fail` outcome. 2. **`+knoe_developer` membership semantics is load-bearing.** A role granted `knoe_developer` is reachable externally; revoking the grant immediately removes external reachability. A role with LOGIN + password but NOT in the group cannot connect externally even with valid creds. (This is the reusability lynchpin from commit `313ee88`; a regression here means "everyone with a postgres password is reachable from the public internet".) 3. **oauth2-proxy `--skip-auth-route` regex tightness.** For each declared route (`/auth/v1/.*`, `/rest/v1/.*`, `/realtime/v1/.*`, `/storage/v1/.*`, `/functions/v1/.*`, `/graphql/v1/.*`, `/pg/.*`, `/onboard.html`, `/support`): - The exact path skips Google sign-in. - Variant paths that *should* be gated (e.g. `/api/profile`, `/Auth/v1/foo`, double-slash, URL-encoded traversal probes) **do** get redirected to Google. - Each declared route is exercised with at least one negative test that would fail if the regex is loosened by accident. 4. **`hostnossl reject` precedence.** A property test: any pg_hba rule order that places a permissive rule (e.g. `host all all 10.0.0.0/8`) before a `hostnossl ... reject` line, and would let an external plaintext connection match the permissive rule, must FAIL the test. This is the regression detector for the `externalTrafficPolicy: Cluster -> Local` bug we hit in Phase 1 — preserve the lesson. ### Tier 2 — Onboarding flow correctness CI-gating (block merges) but not deploy-gating. 5. **`onboard_engineer.sh` is idempotent.** Run twice in a row → one role exists with the second run's password. `--rotate` on a non-existent user → clear error, no partial state. `--revoke` on a connected session → role dropped, sessions terminated cleanly. 6. **`onboard_engineer.sh` input validation.** Username regex; email is `*@knoey.com`; rejection of shell-injection / SQL-injection probes; clear error messages. 7. **Reveal page renders correctly.** Headless-browser tests: - `onboard.html#user=test&pw=Zm9v&exp=2099-01-01T00:00:00Z` → password decoded and shown. - `exp` in the past → expired-state UI. - Missing required fields → error UI. - Copy buttons populate clipboard. - On render, `history.replaceState` strips the fragment from the URL. - The `pw` is base64-decoded correctly even with URL-safe `-_` chars. 8. **CA cert in repo matches live CA.** SHA256 fingerprint of `etc/knoe-db-ca.crt` matches `kubectl get secret knoe-db-ca`. When CNPG rotates, this test fails loudly instead of engineers silently 502'ing. ### Tier 3 — Adjacent surfaces we touched Regression coverage. CI-gating. 9. **Studio internal paths still gate via Google.** `/`, `/api/profile`, `/api/database/*` → 302 to `accounts.google.com`. 10. **`db.0.knoe.dev/storage/v1/bucket` reachable as service-role.** The GRANT we landed in `79a3f76` makes this work; pin it. 11. **`pg_stat_statements` lives in `extensions` post-bootstrap.** PG18 docker fixture for k3d/k3s/min; live-cluster check for gke. Database advisor's `public.pg_stat_statements` warning stays cleared. 12. **`api.0.knoe.dev` alias still works** until [`docs/TODO.md`](../TODO.md) follow-up #14 explicitly retires it. --- ## 4. Decisions you must make in Phase A These are the architectural calls Codex needs nailed down before writing test bodies. Sonnet's reasoning bandwidth is concentrated here. ### 4a. Fixture architecture for "external client" Tier 1 needs to assert "an external client cannot connect with X". Options: | Option | Pros | Cons | |---|---|---| | **k3d cluster + sidecar pod with `externalTrafficPolicy: Local` LB** | Closest to prod (real LB IP, real pg_hba) | Slow to spin up; LB IP allocation in k3d is non-trivial | | **k3d cluster + iptables rule simulating "external" via a labeled namespace's pod IP space** | Faster; deterministic | Diverges from prod; risk that test passes but real LB doesn't | | **Live cluster (gke `knoe-dev-cnpg-0`) + ephemeral test roles** | Highest fidelity | Pollutes prod with test roles; risk of leaving them around; can't run in CI without GCP creds | | **Hybrid**: k3d for most assertions; one nightly job against live cluster for the fidelity-sensitive few | Best of both | Two test rigs to maintain | You decide. Document the tradeoff. Note that CLAUDE.md establishes k3d as the canonical local mode, so the bias should lean k3d unless you find a real reason otherwise. ### 4b. Browser-test framework Tier 2.7 needs headless-browser support. Candidates: Playwright (Node), Puppeteer (Node), Selenium (multi-language), Cypress (Node, e2e-focused). Read [`tests/`](../../tests/) — current Python pytest-only. Adding a Node toolchain just for browser tests is a real cost. Consider: - Are there ~5 browser-tested behaviors total? Then a dedicated framework may be overkill — a pytest test that uses `selenium` (Python) + a headless Chrome may be enough. - Are there ~25? Dedicated framework worth the cost. ### 4c. k3d-per-run vs. shared persistent local cluster | | Per-run | Shared persistent | |---|---|---| | **Speed** | Slow (60-90s spinup) | Fast (~0s) | | **Determinism** | High | Low (tests can pollute each other) | | **Local dev UX** | Friction | Zero friction | | **CI** | Required for parallel runs | Doesn't work for parallel | You decide. Docs/pipeline-phases.md would benefit from your call being recorded explicitly, since Phase 2 of that pipeline will face the same question. ### 4d. Acceptance criteria per tier What counts as "Tier 1 done"? Specifically: - Coverage threshold (e.g. all 36 pg_hba cells; all 9+ skip-auth-route cases)? - Mutation testing pass rate? - Just "the spec items are all green"? Document the answer per tier. **Tier 1's bar should be the highest** — it's the security wall. ### 4e. Retroactive coverage policy The five commits (`f805404` through `313ee88`) landed without tests. We can: - **Cover them all in this round.** The Tier 1 + 2 + 3 spec already covers most of the surface area, but not all (e.g. `pg_stat_statements` schema move's behavior on a NEW deploy isn't directly tested by the spec; the storage GRANT's effect on bucket listing isn't directly tested). - **Cover only the onboarding system in this round; queue the rest for follow-up.** I lean toward "cover them all" — the test debt only grows otherwise — but the call is yours given you can see the actual scope better than I drafted. --- ## 5. Files to read first In order: 1. [`docs/plans/README.md`](README.md) — the team's plan-doc conventions. 2. [`docs/pipeline-phases.md`](../pipeline-phases.md) — where these tests slot into the autobuild pipeline. 3. [`docs/TODO.md`](../TODO.md) — current queue and the f-string blocker you'll fix first. 4. [`tests/test_render_supabase_hostname.py`](../../tests/test_render_supabase_hostname.py) and [`tests/test_repair_update_and_supabase_flags.py`](../../tests/test_repair_update_and_supabase_flags.py) — existing test patterns to follow. 5. [`docs/db-access.md`](../db-access.md) and [`docs/onboarding.md`](../onboarding.md) — what the onboarding system actually does. 6. [`etc/onboard_engineer.sh`](../../etc/onboard_engineer.sh) — the script under test. 7. [`deploy/gcp/gke/knoe-onboard.yaml`](../../deploy/gcp/gke/knoe-onboard.yaml) — the static page (HTML embedded in the ConfigMap). 8. [`deploy/gcp/gke/knoe-db.yaml`](../../deploy/gcp/gke/knoe-db.yaml) — pg_hba block (the surface most needing test coverage). 9. [`deploy/gcp/gke/oauth2-proxy-deployment.yaml`](../../deploy/gcp/gke/oauth2-proxy-deployment.yaml) — `--skip-auth-route` args. 10. [`CLAUDE.md`](../../CLAUDE.md) §"install.sh / deploy.sh pre-flight" — env-contamination warning is a category-mate of the boundary tests you're designing. --- ## 6. Definition of done for Phase A When all four are true: - [ ] `knoe/core/ops/cloudnative_pg.py:1372` f-string fix committed; pytest collection succeeds repo-wide (`pytest --collect-only` returns clean). - [ ] `docs/plans/onboarding-tdd.md` exists, addresses every section in Sec. 2.2 above, and is reviewable by chrisfu. - [ ] The first hard fixture (Sec. 4a's choice, implemented) is committed with at least one Tier 1 test that uses it green. - [ ] You have **not** committed bodies for any other tests. Codex's job. --- ## 7. Out of scope for Phase A (Codex picks up) - Writing test bodies for Tiers 1, 2, 3 (except the one Tier 1 test that validates the fixture from Sec. 4a is functional). - CI wiring beyond what's needed for the f-string fix. - Mutation-testing setup, if your acceptance criteria call for it. - Infrastructure for live-cluster integration tests (only relevant if your Sec. 4a decision calls for one). - Updating engineer-facing docs to mention the test suite. Codex updates `docs/onboarding.md` and `docs/db-access.md` after Phase B lands. --- ## 8. Handoff format to Phase B Codex's brief (separate doc, will be drafted by chrisfu after this lands) will reference: - `docs/plans/onboarding-tdd.md` — the spec Codex implements. - The fixture you committed in Sec. 4a — Codex's tests use it. - Your acceptance criteria per tier — Codex knows when each tier is done. - The directory layout you defined — Codex's PRs land in the right places. So `docs/plans/onboarding-tdd.md` needs to be **prescriptive**, not "considerations" or "options". Codex isn't going to make architectural calls; it'll execute against what you wrote. Land the design. --- ## 9. Working notes - **You can ask chrisfu questions.** If you hit a decision that genuinely needs human input (e.g. "should we let CI provision GCP credentials" — policy call, not technical), surface it rather than guessing. But the questions in Sec. 4 are yours to answer; don't punt them upward. - **The plan in this brief is also wrong somewhere.** The taxonomy is my best draft from a conversation; you may find that some Tier 1 items are redundant with Tier 2, or that there's a Tier 0 ("does the cluster even start?") missing. Calling that out IN the design doc is part of the deliverable. - **Slow is smooth.** This brief deliberately scopes Phase A to design + one fixture so the design isn't rushed. Resist scope creep into Phase B. You're starting fresh in a Sonnet context. Read the files in Sec. 5, then the existing plans in `docs/plans/`, then come back here. Good design lands.