# k3d-mirror-of-GKE — local platform model for knoe-auth dev

> **Active plan. Phase 1 + Phase 2-OIDC-sandbox shipped 2026-05-02;
> Phase 3 in flight (Junie).** Strategic context for a running initiative;
> reads top-to-bottom. Brief refs:
> [Phase 1 (host loop, shipped)](junie/k3d-knoe-auth-dev-loop.md) ·
> [Phase 3 (in-cluster pod, in flight)](junie/k3d-knoe-auth-pod-deploy.md).

---

## 1. Why

knoe-auth Phase 2 (OIDC provider — discovery, authorize, token, userinfo,
JWKS, RS256 signing) just landed on `main` (via the merge that brought
`claude/crazy-bose-fec256` back in). It builds; it hasn't been deployed
to the cluster yet. The next development steps — wiring OAuth2 code
exchange to a persistent table (Phase 2.5), the
ProvisioningWorker→GitLab/Gitea/CNPG dispatch surface, future Round 2
work — all benefit from a tight inner-loop that doesn't require:

- a GKE cluster to push to,
- a deploy pipeline to wait on,
- a real Google OAuth tenant to bounce through, or
- the full supabase + kong + oauth2-proxy + studio stack just to test
  what knoe-auth does.

What we want is a **laptop dev loop**: edit Java, run from IntelliJ
(or `mvn spring-boot:run`), break-pointing freely, against a real
PostgreSQL with the canonical `knoe.*` schema and a real Kerberos KDC
the SPNEGO codepath can talk to.

The four-mode installer we already ship has `k3d` as one of those modes
(see [`deployment-modes.md`](deployment-modes.md)). It brings up CNPG
today. It does **not** yet bring up the bits knoe-auth needs alongside.
This plan closes that gap deliberately, in phases.

## 2. Where the modes stand today

| Mode | DB | knoe-auth | KDC | Supabase | Studio | OAuth2-proxy |
|---|---|---|---|---|---|---|
| `gke` | CNPG (3-replica) | ✅ deployed | ✅ sidecar | ✅ | ✅ | ✅ Google OIDC |
| `k3s` | CNPG (single) | ✅ deployed (manifests at `deploy/opentofu/k3s/manifests/knoe/knoe-auth-*`) | ✅ kdc-configmap | partial | partial | ❌ |
| `k3d` | ✅ CNPG via `k8s/knoe/knoe-db.yaml` | ❌ no manifests | ❌ no manifests | ❌ | ❌ | ❌ |
| `min` | ❌ no Postgres | ❌ | ❌ | ❌ | ❌ | ❌ |

The k3s mode has the auth-stack manifests but is targeted at the
on-prem-VM deploy pattern (talks to a single-node k3s on a customer
machine), not the laptop-dev-loop pattern. Reusing them for k3d is
attractive but the wiring assumptions differ enough (LB types,
ingress class, service-account identity) that copy-and-adapt is
easier than refactor-to-share.

## 3. Strategic shape

The **goal** is not a full GKE mirror — that would mean Studio + Kong
+ supabase + oauth2-proxy + everything, and the laptop fan would not
be happy. The goal is **the smallest k3d-resident stack that lets a
host-resident knoe-auth talk to it end-to-end**.

Decisions (set 2026-05-02 with the user):

1. **knoe-auth runs on the host, not in the cluster.** `mvn
   spring-boot:run` (or IntelliJ Run Configuration) starts the process
   on the laptop. It connects to the k3d cluster's DB and KDC via
   `kubectl port-forward`. This gives:
   - hot reload via Spring DevTools,
   - direct debugger attach,
   - immediate stack traces in the same terminal as the build,
   - no image-build-and-load cycle on every change.

   Trade-off: this isn't testing the deployed-pod-shaped behaviour
   (mounted secrets, SA identity, sidecar networking). That's
   covered by the GKE deploy when it's time to go live, and a future
   "in-cluster smoke" Phase 3 brief if we want pre-merge checks too.

2. **Stack scope is CNPG + KDC only.** No supabase, no Kong, no
   oauth2-proxy, no Studio, no garage/MinIO. knoe-auth doesn't
   depend on any of those for its own surfaces (`/health`,
   `/auth/enroll/*`, `/auth/admin/*`, OIDC discovery/authorize/
   token/userinfo/jwks). The ProvisioningWorker reaches OUT to
   GitLab/Gitea/CNPG — those are mockable or deferrable for dev.

3. **Single-replica CNPG.** Three-replica is GKE-shaped for
   availability; on a laptop it's wasteful. One instance, the same
   PG18 image, same `postInitTemplateSQL` schema bootstrap, same
   `knoe.*` tables.

4. **Kerberos realm `KNOE.LOCAL`.** Distinct from production
   `KNOE.DEV` so credentials don't accidentally cross-talk. Pre-seed
   one admin principal and one developer principal so the engineer
   can `kinit` immediately.

5. **Plain `sslmode=require` for libpq from host.** Production uses
   `verify-full` against the CNPG-issued cert. Locally that needs the
   CA chain and a SAN for `localhost`, which adds yarn-ball complexity.
   `require` keeps TLS on the wire without the cert plumbing. (Engineers
   can opt into `verify-full` later by mounting the CA cert; not
   gated on Phase 1.)

## 4. Architecture (Phase 1 target)

```
   ┌─────────────────────────────────────── Laptop ───────────────────────────────────────┐
   │                                                                                       │
   │   IntelliJ / mvn spring-boot:run                                                      │
   │             │                                                                         │
   │             ▼                                                                         │
   │   knoe-auth   :8080  ──[ http ]──▶  curl / browser tests                             │
   │             │                                                                         │
   │             │  jdbc:postgresql://localhost:5432/knoe-db                                │
   │             │  KRB5_CONFIG=./etc/krb5.local.conf                                      │
   │             │  KDC at  localhost:88                                                    │
   │             │                                                                         │
   │             ▼                                                                         │
   │   ┌─── kubectl port-forward ────┐                                                    │
   │   │   localhost:5432 → svc/knoe-db-rw:5432                                            │
   │   │   localhost:88   → svc/knoe-kdc:88  (TCP + UDP)                                   │
   │   │   localhost:464  → svc/knoe-kdc:464 (kpasswd)                                     │
   │   └────────────────┬─────────────┘                                                   │
   └────────────────────┼─────────────────────────────────────────────────────────────────┘
                        │
                        ▼
   ┌─────────────────────────────────── k3d cluster (single node) ──────────────────────────┐
   │                                                                                          │
   │   ┌──── ns: knoe-db-0 ─────────────┐         ┌──── ns: knoe-system ─────┐                │
   │   │                                 │         │                            │                │
   │   │   CNPG operator                 │         │   knoe-kdc Deployment      │                │
   │   │   knoe-db CNPG Cluster (1 inst) │         │   (krb5kdc + kadmind,      │                │
   │   │     → schema knoe.* via         │         │    realm KNOE.LOCAL,       │                │
   │   │       postInitTemplateSQL       │         │    pre-seeded principals)  │                │
   │   │                                 │         │                            │                │
   │   └─────────────────────────────────┘         └────────────────────────────┘                │
   │                                                                                          │
   └──────────────────────────────────────────────────────────────────────────────────────────┘
```

## 5. Phase 1 scope (Junie brief)

Brief: [`junie/k3d-knoe-auth-dev-loop.md`](junie/k3d-knoe-auth-dev-loop.md).

Deliverables:

1. **CNPG-on-k3d known-good** — a single-replica CNPG cluster with
   the canonical `knoe.*` schema, brought up via a Make target or
   short shell script (`make k3d-knoe-up` style). Reuses existing
   `k8s/knoe/knoe-db.yaml` with the `_strip_k3d_synology_blocks` flow
   the Python orchestrator already runs.
2. **KDC manifest for k3d** — a `Deployment` + `Service` + `ConfigMap`
   under `k8s/knoe/knoe-kdc-*.yaml` (parallel naming to
   `knoe-auth-*`) with realm `KNOE.LOCAL` and a pre-seeded admin
   principal. Init script (`etc/init_knoe_kdc_k3d.sh` or similar)
   adds the principal via `kadmin.local` once the pod is Ready.
3. **Port-forward helper** — a `make k3d-knoe-pf` target (or shell
   script) that opens 5432, 88, 464 to localhost, prints the JDBC
   URL and the `KRB5_CONFIG` env var the engineer should `export`,
   and stays in the foreground until ^C. Background mode optional.
4. **Engineer doc** — `docs/local-dev-knoe-auth.md`: one-time setup,
   daily loop (`make k3d-knoe-up && make k3d-knoe-pf`), how to
   `kinit`, how to point IntelliJ at it, how to reset.
5. **Smoke test** — a shell script that, given the port-forward up,
   verifies: psql connects + `\dt knoe.*` shows ≥6 tables;
   `kadmin.local listprincs` includes the seed admin; from the host,
   `mvn -pl authority test` passes; `mvn -pl authority spring-boot:run`
   starts cleanly and `curl localhost:8080/health` returns `ok` and
   `curl localhost:8080/.well-known/openid-configuration` returns
   valid JSON with the right issuer.

Done = a 5-minute path from `git pull` → editing
`OidcTokenController.java` → re-run → see the change reflected at
`localhost:8080/oauth2/token`, with the DB and KDC backing it real.

## 6. Out of scope for Phase 1 (potential later phases)

- **Phase 2 — KDC: full SPNEGO E2E** including a host-side
  `firefox`/`curl` that does negotiate against `localhost:8080`.
  Needs `KRB5_CONFIG`, an SPN like `HTTP/localhost@KNOE.LOCAL`, a
  keytab file mounted into knoe-auth's runtime. Phase 1 stops at the
  KDC being reachable; Phase 2 makes SPNEGO actually work.
- **Phase 3 — image-build-and-load smoke** ([brief](junie/k3d-knoe-auth-pod-deploy.md),
  in flight 2026-05-02). Engineers run `make k3d-knoe-deploy` and see
  knoe-auth as a pod in the cluster, exercising image build, mounted
  secrets, and KDC-sidecar wiring without pushing to GKE. Pre-merge
  gate; not required for daily dev.
- **Phase 4 — supabase stack on k3d** for end-to-end SDK testing
  against a local DB. Heavy. Only if/when someone needs it.
- **Phase 5 — oauth2-proxy + Google OIDC on localhost** for testing
  the Studio gate flow without going to GKE. Even heavier; needs a
  Google OAuth client with `localhost` redirect URLs registered.
  Probably never worth it.
- **Phase 2.5 (separate track) — persistent OAuth2 code/session
  tables.** Currently `OidcCodeService` uses an in-memory
  `ConcurrentHashMap`, which means codes don't survive a knoe-auth
  restart. Not blocked by k3d work but useful to have once dev
  iteration is fast enough that engineers exercise the flow often.
  Tracked separately; would touch `authority/src/main/.../session/`
  and add migrations under `authority/src/main/resources/db/migration/`.

## 7. Risks / open questions

- **Resource ceiling on a MacBook.** A single-replica CNPG plus KDC
  is light (~600 MB RAM combined), but k3d itself + Docker Desktop
  + IntelliJ + Spring Boot can climb. Document a "low-RAM mode" if
  it becomes a real problem; don't speculate now.
- **k3d's Docker networking and `kubectl port-forward` UDP.** UDP
  port-forward (port 88 for Kerberos UDP, 464 for kpasswd) works in
  recent kubectl versions but has historically been finicky. If it
  fails, fall back to TCP-only Kerberos (`tcp_only` in `krb5.conf`).
  Phase 1 should set this in the engineer-side `krb5.local.conf`
  preemptively.
- **CNPG operator CRD installation in fresh k3d.** The `make k3d-up`
  flow today runs `init_cnpg_gke.sh` for GKE, but there's no
  symmetric `init_cnpg_k3d.sh`. Either: (a) Junie writes a thin one
  that just `kubectl apply -f`s the CNPG release manifest with no
  GCP/IAM steps, or (b) the existing Python orchestrator
  `knoe.core.ops.cloudnative_pg` already handles k3d (per the
  survey, it does — `_strip_k3d_synology_blocks` is in there).
  Verify which during Phase 1.
- **Schema bootstrap.** The GKE flow seeds `knoe.*` via
  `etc/init_knoe_auth.sh` running SQL against the CNPG primary
  through `kubectl exec`. The same approach works in k3d; the
  brief should reuse the script (with a `KUBECONTEXT` swap), not
  duplicate it.

## 8. Verification (when Phase 1 is "done")

A new engineer, freshly cloning the repo on a Mac:

```bash
git clone <repo> && cd knoe-db
make k3d-knoe-up                # provisions cluster, CNPG, KDC, schema
make k3d-knoe-pf &              # port-forward in background
mvn -pl authority spring-boot:run
# in another terminal:
curl http://localhost:8080/health
# → ok
curl http://localhost:8080/.well-known/openid-configuration
# → JSON, issuer "http://localhost:8080"
```

Total elapsed time on a fresh laptop: ≤ 8 minutes. The engineer can
now edit `authority/src/main/java/dev/knoe/auth/...` and see changes
without leaving their editor.

## 9. Glossary

- **k3d** — `k3s` running inside a Docker container. Lightweight
  Kubernetes for laptops; no VM needed.
- **CNPG** — `cloudnative-pg`, a PostgreSQL Kubernetes operator. We
  run a single instance in k3d, three in GKE.
- **KDC** — Key Distribution Center, the Kerberos protocol's auth
  server. SPNEGO browsers/clients fetch tickets from it; knoe-auth
  validates SPNEGO challenges using these.
- **KNOE.LOCAL** — the Kerberos realm name we use for laptop dev.
  Distinct from `KNOE.DEV` (production) so a stray ticket from one
  doesn't authenticate you on the other.
- **SPNEGO** — `Negotiate` HTTP auth scheme. Carries a Kerberos
  ticket inside an HTTP header.
- **Phase 2.5** — colloquial name for the gap between Phase 2 OIDC
  (in-memory codes) and a fully persistent OIDC provider (DB-backed
  codes/sessions). Not formally numbered; will get a name when work
  starts.