# knoe-system — knoe-auth deployment and k3d TDD guide

Single reference for the knoe-auth identity service: what it does, how it is
deployed on the GKE `knoe-db` stack, and how to spin up a local k3d cluster
for test-driven development.

---

## 1. What knoe-auth is

`knoe-auth` is the identity system for the knoe.dev platform. It decides who
you are, what you can access, and how new contributors come on board.

**Round 1 (shipped)** — Durable Kerberos identity + contributor enrollment.
A Spring Boot service (`authority/`) wraps a KDC sidecar. Engineers enroll
via an invite OTP → Google corroboration → TOTP setup flow; the result is a
Kerberos principal (`alice@KNOE.DEV`) and a `knoe.user` row in PostgreSQL.
Downstream provisioning (GitLab, Gitea, CNPG roles) is queued automatically.

**Phase 2 (planned, not yet implemented)** — knoe-auth becomes a full OIDC
provider (`issuer: https://api.knoe.dev/auth`). Services that currently trust
Google directly will trust knoe-auth instead; knoe-auth federates upstream to
Google. See [`docs/knoe-auth-phase-2.md`](knoe-auth-phase-2.md) and the
paused `pg_oauth` item in [`docs/TODO.md`](TODO.md).

### Trust model (Round 1)

The invite OTP is the trust anchor. Google sign-in is corroboration. TOTP is
the ongoing second factor. knoe.dev never pre-configures trust with any
external Google Workspace — any verified Google account works once invited.

---

## 2. Work-in-progress status

| Item | Status | Notes |
|---|---|---|
| Round 1 Kerberos enrollment | **Shipped** | Operational on GKE |
| k3d dev loop (Phase 1 brief) | **Shipped** | `make k3d-knoe-up` |
| Phase 2 OIDC provider — **k3d setup** | **Shipped** | `application-k3d.yml` + `etc/gen_oidc_signing_key.sh`; OIDC discovery / JWKS / token endpoints exercise locally. See [`docs/local-dev-knoe-auth.md`](local-dev-knoe-auth.md) §"Daily loop". |
| Phase 2 OIDC provider — **GKE deploy** | **Shipped** | `KNOE_AUTH_OIDC_ENABLED=true` set; Kong `/auth` route added; signing key wired via `knoe-auth-oidc` secret (1Password). Run `./etc/init_knoe_auth.sh initialize` to create the secret. |
| pg_oauth (PG18 OAUTHBEARER) | **Ready** | OIDC issuer shipped to GKE; pg_oauth can resume |
| Round 1.5 OpenBao transit-key encryption | **Later** | `UserProvisioningService.java:84` TODO |

---

## 3. GKE deployment — knoe-auth on knoe-db

### Clusters

| Resource | Cluster | Namespace |
|---|---|---|
| knoe-auth pod (Spring Boot + KDC sidecar) | `knoe-dev-0` (app cluster) | `knoe-system` |
| PostgreSQL (CNPG `knoe-db`) | `knoe-dev-cnpg-0` (DB cluster) | `knoe-db-0` |

knoe-auth connects to PostgreSQL over the internal service
`knoe-db-rw.knoe-db-0.svc.cluster.local:5432` (cross-cluster via VPC-native
routing; both clusters share the same VPC in `us-west3`).

### Key manifests

| File | Purpose |
|---|---|
| `deploy/gcp/gke/knoe-auth-deployment.yaml` | Deployment: KDC sidecar + keytab-bootstrap initContainer + Spring Boot container. Realm `KNOE.DEV`. Image: `knoe-auth:latest`. |
| `deploy/gcp/gke/knoe-kdc-configmap.yaml` | `krb5.conf` + `kdc.conf` for realm `KNOE.DEV`. |
| `deploy/gcp/gke/knoe-kdc-secrets.yaml` | Master key + admin password. Production values come from OpenBao; file is a placeholder only — no real secrets committed. |
| `deploy/gcp/gke/knoe-auth-google-oidc-secret.example.yaml` | `GOOGLE_CLIENT_ID` / `GOOGLE_CLIENT_SECRET` template. |
| `deploy/gcp/gke/workload-identity.yaml` | KSA↔GSA bindings for GCP secret access. |
| `deploy/gcp/gke/knoe-db.yaml` | CNPG Cluster manifest (3-replica, GCS barman backup). |

### Init script

`etc/init_knoe_auth.sh` provisions the `knoe.*` schema and seeds initial
state. Run it after applying the manifests:

```bash
# GKE (default mode — requires kubectl context pointing at knoe-dev-0)
./etc/init_knoe_auth.sh schema
./etc/init_knoe_auth.sh status
```

Subcommands: `schema` (create/migrate tables), `status` (print principal +
table counts), `invite <email>` (issue an enrollment invite).

### Environment variables (Spring Boot container)

| Variable | Purpose |
|---|---|
| `KNOE_KDC_REALM` | Kerberos realm (default `KNOE.DEV`) |
| `KNOE_KERBEROS_SERVICE_PRINCIPAL` | Service principal for SPNEGO (`HTTP/auth.knoe.dev@KNOE.DEV`) |
| `KNOE_AUTH_BASE_URL` | Public base URL (`https://api.knoe.dev/auth`) |
| `KNOE_AUTH_COOKIE_DOMAIN` | Cookie scope (`knoe.dev`) |
| `KNOE_AUTH_SESSION_SECRET` | HMAC key for session JWTs |
| `KNOE_AUTH_OIDC_ENABLED` | `false` until Phase 2 ships |
| `KNOE_AUTH_OIDC_ISSUER` | Phase 2 issuer URL |
| `KNOE_AUTH_OIDC_SIGNING_KEY` | Base64 PKCS#8 RS256 private key (Phase 2) |
| `KNOE_AUTH_GOOGLE_CLIENT_ID` | Upstream Google OAuth client |
| `KNOE_AUTH_GOOGLE_CLIENT_SECRET` | Upstream Google OAuth secret |

### Apply sequence (fresh GKE deploy)

```bash
# 1. Ensure kubectl context is knoe-dev-0
kubectl config use-context gke_plenary-truck-485623-p7_us-west3_knoe-dev-0

# 2. Apply KDC config + secrets (secrets must be pre-populated from OpenBao)
kubectl apply -f deploy/gcp/gke/knoe-kdc-configmap.yaml
kubectl apply -f deploy/gcp/gke/knoe-kdc-secrets.yaml   # populate first!

# 3. Apply the deployment
kubectl apply -f deploy/gcp/gke/knoe-auth-deployment.yaml

# 4. Wait for rollout
kubectl -n knoe-system rollout status deployment/knoe-auth

# 5. Seed schema (runs against the DB cluster via the pod's psql)
./etc/init_knoe_auth.sh schema
./etc/init_knoe_auth.sh status
```

---

## 4. k3d local dev loop (TDD)

Run knoe-auth from IntelliJ or `mvn spring-boot:run` against a real
PostgreSQL + Kerberos KDC, both in a local k3d cluster. No GKE, no
1Password, no Google OAuth tenant required.

Full details: [`docs/local-dev-knoe-auth.md`](local-dev-knoe-auth.md).
Quick reference below.

### Prerequisites (install once)

```bash
brew install k3d kubectl maven openjdk@21
# Docker Desktop must be running
```

Verify:

```bash
k3d version        # ≥ 5.x
kubectl version --client
mvn -version       # Maven 3.x, Java 21
docker info        # must not error
```

### One-time cluster setup

```bash
make k3d-knoe-up
```

Takes 3–5 minutes. Creates a single-node k3d cluster (`k3d-knoe`), installs
the CNPG operator (v1.29.0), applies a single-replica `knoe-db` PostgreSQL
cluster, deploys the KDC (realm `KNOE.LOCAL`), seeds the `knoe.*` schema,
and seeds the `knoe_developer` group role + per-engineer LOGIN role
**`chrisfu`** (password `chrisfu-dev`, member of `knoe_developer`) so the
host can connect as `chrisfu@knoey.com` over the port-forward.

Verify:

```bash
kubectl --context=k3d-k3d-knoe -n knoe-db-0 get cluster knoe-db
# → knoe-db   Cluster in healthy state

make k3d-knoe-smoke   # full smoke check
```

### Daily dev loop

```bash
# Terminal 1 — keep port-forwards alive
make k3d-knoe-pf
# Opens: localhost:5432 → PostgreSQL, localhost:88 → KDC (TCP), localhost:464 → kpasswd

# Terminal 2 — get a Kerberos ticket
export KRB5_CONFIG=$(pwd)/etc/krb5.local.conf
kinit developer@KNOE.LOCAL          # password: developer
klist                                # confirm ticket

# Terminal 3 — run knoe-auth
cd authority
mvn spring-boot:run \
  -Dspring-boot.run.jvmArguments="-Djava.security.krb5.conf=$(pwd)/../etc/krb5.local.conf" \
  -Dspring-boot.run.profiles=k3d

# Verify
curl http://localhost:8080/health    # → {"status":"ok"}
```

### IntelliJ run config

Add a Spring Boot run configuration for `KnoeAuthApplication` with:

- **VM options:** `-Djava.security.krb5.conf=$PROJECT_DIR$/etc/krb5.local.conf`
- **Active profiles:** `k3d`
- **Working directory:** `$MODULE_WORKING_DIR$`

### Kerberos realm differences

| | k3d (local) | GKE (production) |
|---|---|---|
| Realm | `KNOE.LOCAL` | `KNOE.DEV` |
| KDC address | `localhost:88` (port-forwarded) | `knoe-kdc.knoe-system.svc` |
| Admin principal | `admin/admin@KNOE.LOCAL` | OpenBao-managed |
| Developer principal | `developer@KNOE.LOCAL` (password: `developer`) | per-engineer invite flow |
| krb5 config | `etc/krb5.local.conf` | pod-mounted from `knoe-kdc-configmap.yaml` |

`etc/krb5.local.conf` sets `udp_preference_limit = 1` to force TCP — required
because `kubectl port-forward` does not proxy UDP on macOS.

### Tear down

```bash
make k3d-knoe-down   # deletes the k3d cluster entirely
```

### Smoke check details

`scripts/k3d-knoe-smoke.sh` verifies:

1. CNPG cluster `knoe-db` is in `Cluster in healthy state`
2. At least 6 `knoe.*` tables exist in PostgreSQL
3. KDC Deployment is `1/1 Ready`
4. `admin/admin@KNOE.LOCAL` principal exists in the KDC database

---

## 5. Schema overview

The `knoe.*` schema lives in the CNPG PostgreSQL cluster (`knoe-db-0`
namespace, database `knoe`). Core tables:

| Table | Purpose |
|---|---|
| `knoe.user` | Canonical user record (id, username, email, created_at) |
| `knoe.identity` | Per-provider identity rows (google, kerberos) linked to `knoe.user` |
| `knoe.invitation` | Pending invites; OTP stored as bcrypt hash |
| `knoe.totp_credential` | TOTP secrets per user (AES-GCM encrypted at rest; Round 1.5 will add OpenBao transit-key envelope) |
| `knoe.knobject` | Provisioned resources (GitLab project, Gitea repo, CNPG role, …) |
| `knoe.access_grant` | User↔knobject grants |
| `knoe.provisioning_job` | Async job queue polled by `ProvisioningWorker` |

Schema is applied/migrated by `etc/init_knoe_auth.sh schema` in both GKE and
k3d modes.

---

## 6. Source map

### Java application (`authority/`)

| File | Responsibility |
|---|---|
| `KnoeAuthApplication.java` | `@SpringBootApplication` entry point |
| `HealthController.java` | `GET /health` |
| `web/LoginController.java` | Form-login + SPNEGO challenge |
| `web/VerifyController.java` | Token-verify endpoint for downstream services |
| `session/SessionTokenService.java` | HMAC-SHA256 JWT cookies |
| `kerberos/KerberosSpnegoService.java` | SPNEGO challenge/response |
| `kerberos/KadminClient.java` | Shells out to `kadmin.local` to addprinc/cpw (input sanitized) |
| `enroll/EnrollmentController.java` | Enrollment web flow (OTP → Google → TOTP → complete) |
| `enroll/InviteService.java` | Invite CRUD, OTP bcrypt hashing, rate limiting |
| `enroll/UserProvisioningService.java` | Transactional orchestrator: inserts rows, calls KadminClient, queues jobs |
| `admin/AdminController.java` | `POST /auth/admin/invites`, `GET /auth/admin/users` — SPNEGO + admin-role gated |
| `provisioning/ProvisioningWorker.java` | `@Scheduled` poller for `knoe.provisioning_job WHERE status='pending'` |
| `config/AuthProperties.java` | Typed binding for `knoe.auth.*` config keys |

### k3d manifests (`k8s/knoe/`)

| File | Purpose |
|---|---|
| `knoe-db.yaml` | CNPG Cluster (single-replica, `local-path` storage) |
| `knoe-kdc-configmap.yaml` | KDC `krb5.conf` + `kdc.conf`, realm `KNOE.LOCAL` |
| `knoe-kdc-deployment.yaml` | KDC Deployment (debian:bookworm-slim + MIT Kerberos) |
| `knoe-kdc-service.yaml` | ClusterIP service for KDC (ports 88, 464) |
| `knoe-kdc-pvc.yaml` | PVC for KDC database persistence |
| `knoe-kdc-init-job.yaml` | One-shot Job: creates `admin/admin` + `developer` principals |

---

## 7. Open work items

- **pg_oauth (Phase 2 pg_hba)** — Ready to resume. Replaces SCRAM with PG18
  native OAUTHBEARER; wires into `install.sh` / `supabase/deploy.sh`.
  OIDC issuer now reachable at `https://api.knoe.dev/auth` — pg_oauth can proceed.

- **Round 1.5 OpenBao transit-key encryption** — `UserProvisioningService.java:84`
  has the TODO. TOTP secrets currently have PG-level encryption only.