prole/docs/local-dev-knoe-auth.md

Local dev loop for knoe-auth

Run knoe-auth from IntelliJ or mvn spring-boot:run against a real PostgreSQL + Kerberos KDC, both running in a local k3d cluster. No GKE cluster, no deploy pipeline, no Google OAuth tenant needed.

---

Prerequisites

Install once on your Mac:

brew install k3d kubectl maven openjdk@21

Also required: Docker Desktop running (k3d uses it for the cluster node).

Verify:

k3d version        # ≥ 5.x
kubectl version --client
mvn -version       # Apache Maven 3.x, Java 21
docker info        # must not error

---

One-time setup

git clone <repo> && cd knoe-db
make k3d-knoe-up

This takes 3–5 minutes on a fresh laptop. It:

1. Creates a single-node k3d cluster named k3d-knoe
2. Installs the CNPG operator (v1.29.0)
3. Applies a single-replica knoe-db CNPG Cluster (PostgreSQL 17)
4. Waits for the cluster to reach Cluster in healthy state
5. Applies the KDC Deployment + Service (realm KNOE.LOCAL)
6. Seeds the knoe.* schema via etc/init_knoe_auth.sh schema --mode k3d

Verify it worked:

kubectl --context=k3d-k3d-knoe -n knoe-db-0 get cluster knoe-db
# → knoe-db   Cluster in healthy state

kubectl --context=k3d-k3d-knoe -n knoe-system get deploy knoe-kdc
# → knoe-kdc   1/1   ...

scripts/k3d-knoe-smoke.sh
# → k3d-knoe-smoke: PASS

---

Daily loop

Open two terminals from the repo root.

Terminal A — port-forwards (leave running)

make k3d-knoe-pf

This opens:

Local port	Destination	Purpose
5432	knoe-db-rw (knoe-db-0)	PostgreSQL primary
88	knoe-kdc (knoe-system)	Kerberos KDC (TCP)
464	knoe-kdc (knoe-system)	kpasswd (TCP)

The script prints the JDBC URL and env vars to export. Leave it running; press ^C to stop.

Terminal B — knoe-auth

export KRB5_CONFIG=$PWD/etc/krb5.local.conf
export SPRING_DATASOURCE_URL=jdbc:postgresql://localhost:5432/knoe-db?sslmode=require
export SPRING_DATASOURCE_USERNAME=knoe
export SPRING_DATASOURCE_PASSWORD=knoe-dev-password

# Phase 2 OIDC signing key — generated by `make k3d-knoe-up` (idempotent;
# persisted at etc/secrets/knoe-auth-oidc-key.b64). If absent, OidcTokenService
# falls back to an ephemeral key (loops still work; tokens just don't survive
# a knoe-auth restart).
export KNOE_AUTH_OIDC_SIGNING_KEY=$(cat etc/secrets/knoe-auth-oidc-key.b64)

mvn -pl authority spring-boot:run \
  -Dspring-boot.run.jvmArguments="-Djava.security.krb5.conf=$PWD/etc/krb5.local.conf" \
  -Dspring-boot.run.profiles=k3d

The k3d Spring profile (authority/src/main/resources/application-k3d.yml) enables Phase 2 OIDC, points the issuer at http://localhost:8080, sets the Kerberos realm to KNOE.LOCAL, and aligns the datasource defaults with the port-forwards.

Verify the loop

curl http://localhost:8080/health
# → {"status":"ok"}

curl -s http://localhost:8080/.well-known/openid-configuration | jq .issuer
# → "http://localhost:8080"

curl -s http://localhost:8080/jwks.json | jq '.keys[0].kty'
# → "RSA"

If /.well-known/openid-configuration returns 404, OIDC isn't enabled — verify you started with -Dspring-boot.run.profiles=k3d (or that the k3d profile is active in your IntelliJ run config).

---

Verify psql connectivity

make k3d-knoe-up seeds two roles for local dev (in addition to the CNPG-internal postgres/knoe superusers):

Role	Password	Group	Purpose
chrisfu	chrisfu-dev	knoe_developer	Per-engineer LOGIN role; mirrors the GKE pattern (see docs/db-access.md). Use this for daily psql/JDBC work as chrisfu@knoey.com.
knoe_developer	(none — NOLOGIN)	—	Group role with R/W on knoe+public. Granted to chrisfu. Add new dev users with GRANT knoe_developer TO <user>.

With make k3d-knoe-pf running:

# Connect as chrisfu (the engineer identity)
PGPASSWORD=chrisfu-dev psql \
  "postgresql://chrisfu@localhost:5432/knoe-db?sslmode=require" \
  -c "\dt knoe.*"

# → ≥6 tables: knoe.invitation, knoe.identity, knoe.totp_credential,
#   knoe.knobject, knoe.access_grant, knoe.provisioning_job (+ knoe.user)

Same connection string for IntelliJ DataGrip / DBeaver — host localhost, port 5432, db knoe-db, user chrisfu, password chrisfu-dev, sslmode require.

Why chrisfu? The k3d seed sets up the same identity layout as production (where chrisfu and ron are the human engineer roles). Re-running make k3d-knoe-up is idempotent — the password is reset to chrisfu-dev on every up, so you always know what to use after a rebuild.

---

kinit for Kerberos testing (Phase 1 — KDC reachable)

With make k3d-knoe-pf running and KRB5_CONFIG exported:

kinit admin/admin@KNOE.LOCAL
# Password: knoe-local-admin-dev
klist
# → Credentials cache: ...  admin/admin@KNOE.LOCAL

Phase 1 scope: The KDC is reachable and principals exist. Full SPNEGO E2E (browser Negotiate header → knoe-auth) requires a service principal keytab for HTTP/localhost@KNOE.LOCAL — that's Phase 2.

---

IntelliJ run configuration

1. Open the project in IntelliJ.
2. Run → Edit Configurations → + → Spring Boot
3. Set:
   • Module: authority
   • Main class: dev.knoe.auth.KnoeAuthApplication
   • Active profiles: k3d (enables Phase 2 OIDC + dev-only defaults)
   • VM options: -Djava.security.krb5.conf=$PROJECT_DIR$/etc/krb5.local.conf
4. Under Environment variables, add:

KRB5_CONFIG=/path/to/knoe-db/etc/krb5.local.conf
SPRING_DATASOURCE_URL=jdbc:postgresql://localhost:5432/knoe-db?sslmode=require
SPRING_DATASOURCE_USERNAME=knoe
SPRING_DATASOURCE_PASSWORD=knoe-dev-password
KNOE_AUTH_OIDC_SIGNING_KEY=<paste contents of etc/secrets/knoe-auth-oidc-key.b64>

IntelliJ's environment-variable field doesn't expand $(cat …) shell substitution, so paste the base64 string directly (it's one line, ~1.6 KB). Or use a .env plugin like EnvFile and point it at the .b64 file.

5. Click OK, then Run.

Spring DevTools is active — saving a .java file triggers a hot reload automatically.

---

Reset / rebuild loop

The whole stack is designed to be repeatedly rebuilt. Each make k3d-knoe-up is idempotent:

• Cluster name, namespaces, secrets, schema, and dev-user roles (chrisfu with password chrisfu-dev, member of knoe_developer) are all recreated from scratch.
• The OIDC RS256 keypair at etc/secrets/knoe-auth-oidc-key.b64 is kept across rebuilds (so JWTs you issued before a rebuild still validate). Pass FORCE=1 bash etc/gen_oidc_signing_key.sh if you want to rotate.

Two cadences:

# Full rebuild (~3-5 min) — wipes the k3d cluster, recreates everything.
# Use after schema changes, manifest edits, or when in doubt.
make k3d-knoe-down && make k3d-knoe-up

# Mid-iteration: knoe-auth restart only.
# In Terminal B (the one running mvn), just ^C and re-run:
mvn -pl authority spring-boot:run \
  -Dspring-boot.run.jvmArguments="-Djava.security.krb5.conf=$PWD/etc/krb5.local.conf" \
  -Dspring-boot.run.profiles=k3d
# Spring DevTools also picks up `.java` saves automatically — no manual restart needed.

If you only changed the schema or want to reseed roles without rebuilding the cluster:

./etc/init_knoe_auth.sh schema --mode k3d        # re-runs the schema SQL
./etc/init_knoe_auth.sh initialize --mode k3d    # full re-init incl. dev-user seed

These are safe to run while the cluster's up — both use IF NOT EXISTS / ALTER ROLE patterns and are non-destructive to existing data (apart from resetting chrisfu's password back to chrisfu-dev).

---

Troubleshooting

make k3d-knoe-up fails at CNPG operator install

Check Docker Desktop is running and has network access:

docker info
curl -I https://raw.githubusercontent.com/cloudnative-pg/cloudnative-pg/release-1.29/releases/cnpg-1.29.0.yaml

CNPG cluster stuck in Setting up primary

The PostgreSQL image pull may be slow on first run. Watch:

kubectl --context=k3d-k3d-knoe -n knoe-db-0 get pods -w

Port 5432 already in use

Stop any local PostgreSQL instance:

brew services stop postgresql@17   # or whatever version

Or run the port-forward on a different port:

kubectl --context=k3d-k3d-knoe -n knoe-db-0 port-forward service/knoe-db-rw 15432:5432
# then set SPRING_DATASOURCE_URL=jdbc:postgresql://localhost:15432/knoe-db?sslmode=require

kinit fails with Cannot contact any KDC

Ensure make k3d-knoe-pf is running and KRB5_CONFIG is exported:

echo $KRB5_CONFIG   # should be .../etc/krb5.local.conf
# If empty:
export KRB5_CONFIG=$PWD/etc/krb5.local.conf
kinit admin/admin@KNOE.LOCAL

The udp_preference_limit = 1 in krb5.local.conf forces TCP, which works reliably through kubectl port-forward on macOS.

Schema not applied (knoe.* tables missing)

Re-run the schema step manually:

etc/init_knoe_auth.sh schema --mode k3d

---

Architecture reference

See docs/plans/k3d-gke-mirror.md for the multi-phase plan and the "why" behind this setup.

Phase 1 (this doc): CNPG + KDC in k3d, knoe-auth on host.
Phase 2 (future): Full SPNEGO E2E with keytab.
Phase 3 (future): knoe-auth as a pod inside k3d (image build/load).