# knoe-auth — prole.org deployment overlay

> **Customer-specific overlay** of the canonical architecture
> documented in [`~/dev/knoe-db/docs/knoe-auth.md`](https://github.com/dredx/knoe-db/blob/main/docs/knoe-auth.md).
> Read the canonical doc first for components, flows, and intent.
> This file documents what's specific to the **`prole.org` homelab
> deployment** — hostnames, IdP wiring, Kerberos realm names, cross-realm
> trust setup, current state of each integration.
>
> Filed 2026-05-11. Related: [`db-access.md`](db-access.md),
> [`knoe-auth-phase-2.md`](knoe-auth-phase-2.md),
> [`oidc-setup.md`](oidc-setup.md),
> [`local-dev-knoe-auth.md`](local-dev-knoe-auth.md).

---

## 1. Identity sources (what's authoritative)

The prole deployment has **two upstream identity layers**, each
authoritative for a different population:

| IdP | Authoritative for | Realm / Domain |
|---|---|---|
| **Samba AD on myrddin** | Homelab humans + workstation logins | Kerberos realm `PROLE.ORG`; SMB workgroup `PROLE` |
| **Google Workspace (knoey.com)** | Engineers, OIDC sign-ins, email-based identity | `@knoey.com` |

Both feed into `knoe-auth` as upstream IdPs:

- Samba AD via **Kerberos cross-realm trust** (KNOE.LOCAL ↔ PROLE.ORG)
- Google Workspace via **OIDC** (knoe-auth registers as a Google OAuth client)

A single human (chrisfu, ron, …) typically has both:
`chrisfu@PROLE.ORG` (workstation Kerberos identity) and
`chrisfu@knoey.com` (Google Workspace / OIDC identity). The
`preferred_username` claim canonicalizes them to the same short-name
(`chrisfu`) so downstream services see one identity.

## 2. Deployments

Two clusters in the prole homelab, plus the managed GKE deployment
for production:

| Cluster | Where | Role | knoe-auth URL |
|---|---|---|---|
| **k3s** (`prole-service-cluster`) | myrddin + merlin + gandalf | Homelab production — Gitea, Supabase, OpenBao, Grafana, Garage | `https://auth.prole.org` (planned) / `http://knoe-auth.knoe-system.svc.cluster.local:8080` (in-cluster) |
| **k3d** (`k3d-knoe`) | Engineer laptop | Dev loop — knoe-auth Phase 3 shipped | `http://localhost:8090` (host port-forward) |
| **GKE** (`knoe-dev-0` + `knoe-dev-cnpg-0`) | `plenary-truck-485623-p7` | Production OIDC issuer for the broader `knoe.dev` platform | `https://api.knoe.dev/auth` |

A workstation in the prole homelab typically uses **the GKE-hosted
issuer** for OIDC (because that's where Google Workspace OAuth client
registration lives) and **the k3s in-cluster KDC** for Kerberos
(because that's where the Samba AD trust lives). Both are addressable
externally via their respective ingresses.

## 3. Hostname map

| Hostname | Resolves to | What lives there |
|---|---|---|
| `myrddin.prole.org` | LAN IP | Samba AD DC, k3s server node, Kerberos KDC for `PROLE.ORG` |
| `merlin.prole.org`, `gandalf.prole.org` | LAN IPs | k3s agent nodes |
| `morgana.prole.org`, `zinfandel.prole.org` | LAN IPs | Engineer workstations |
| `git.prole.org` | k3s Traefik | Gitea + SPNEGO proxy |
| `db.prole.org` | k3s Traefik | Supabase Studio (current: Google OAuth via oauth2-proxy) |
| `auth.prole.org` | k3s Traefik (planned) | knoe-auth OIDC issuer (homelab path) |
| `svc.prole.org` | k3s Traefik | Kong API gateway for internal services |
| `api.knoe.dev` | GKE GCE LB | knoe-auth OIDC issuer (production path) |
| `db.knoe.dev` | GKE GCE LB (planned) | Supabase Studio + knoe-auth OIDC (per [`per-user-supabase-workspace.md`](../docs/plans/junie/per-user-supabase-workspace.md)) |
| `db.0.knoe.dev` | GKE GCE LB | Supabase Studio (current; Google OAuth via oauth2-proxy) — to be aliased + deprecated when `db.knoe.dev` lands |
| `pg.0.knoe.dev` | GKE TCP LB | CNPG external endpoint (port 5432) |

## 4. Cross-realm Kerberos trust (PROLE.ORG ↔ KNOE.LOCAL)

This is the prole-specific glue that lets a workstation user with a
PROLE.ORG TGT obtain a KNOE.LOCAL service ticket without re-typing
a password.

### 4.1 Setup

Provisioned by the Ansible playbook
[`infrastructure/playbooks/kerberos_trust_setup.yml`](../infrastructure/playbooks/kerberos_trust_setup.yml)
(landed in commits `5cece40`..`ad1eced` on 2026-05-10/11). Reads the
shared `trust_shared_password` from the k8s Secret
`knoe-system/knoe-kdc-secrets` and writes it both:

- **On myrddin (Samba side)** — creates user `krbtgt_KNOE.LOCAL` in
  Samba with the shared password, sets UPN to
  `krbtgt/KNOE.LOCAL@PROLE.ORG` (the Kerberos-canonical form), pins
  `msDS-SupportedEncryptionTypes = 4` (RC4 only).
- **In-cluster MIT KDC** — `etc/init_kdc.sh` creates both
  `krbtgt/KNOE.LOCAL@PROLE.ORG` and `krbtgt/PROLE.ORG@KNOE.LOCAL`
  locally via `kadmin.local addprinc -e arcfour-hmac:normal` with the
  matching shared password.

RC4 enforcement on both sides is critical — AES key derivation uses a
salt (Samba salts as `<remote_realm>+UPN`, MIT salts as
`<local_realm>+<principal-no-realm>`), so the same password produces
different AES keys on each side. RC4 has no salt, keys converge.

### 4.2 What works today

Verified 2026-05-10 from morgana (workstation) and myrddin (DC):

```
$ kdestroy && kinit chrisfu@PROLE.ORG
$ kvno krbtgt/KNOE.LOCAL@PROLE.ORG
krbtgt/KNOE.LOCAL@PROLE.ORG: kvno = 3
$ klist
Ticket cache: FILE:/tmp/krb5cc_1000
Default principal: chrisfu@PROLE.ORG
  krbtgt/PROLE.ORG@PROLE.ORG     (local TGT)
  krbtgt/KNOE.LOCAL@PROLE.ORG    (cross-realm referral)
```

Workstation Kerberos config is provisioned by
[`infrastructure/playbooks/workstation_kerberos.yml`](../infrastructure/playbooks/workstation_kerberos.yml)
(landed in earlier commits). It deploys `/etc/krb5.conf` with both
realms defined, plus Chrome/Edge managed-policy `AuthServerAllowlist`
so the browser will offer Kerberos SPNEGO for `*.prole.org`.

### 4.3 Cross-realm flow

```mermaid
sequenceDiagram
    autonumber
    participant WS as Workstation<br/>(morgana, PROLE.ORG-joined)
    participant SambaKDC as Samba KDC<br/>(myrddin, PROLE.ORG)
    participant MITKDC as MIT KDC<br/>(in-cluster, KNOE.LOCAL)
    participant Svc as Kerberized service<br/>(HTTP/git.prole.org)

    WS->>SambaKDC: AS_REQ chrisfu@PROLE.ORG
    SambaKDC-->>WS: TGT (krbtgt/PROLE.ORG@PROLE.ORG)
    Note over WS: User opens https://git.prole.org<br/>(realm of svc = PROLE.ORG for SPNEGO,<br/>but pretend = KNOE.LOCAL for illustration)
    WS->>SambaKDC: TGS_REQ for krbtgt/KNOE.LOCAL@PROLE.ORG
    Note right of SambaKDC: Samba's krbtgt_KNOE.LOCAL user<br/>has the shared RC4 key
    SambaKDC-->>WS: cross-realm TGT<br/>(enc with shared key, kvno N)
    WS->>MITKDC: TGS_REQ for HTTP/[svc-host]@KNOE.LOCAL
    Note right of MITKDC: Decrypts inbound TGT<br/>with matching RC4 key (kvno N)
    MITKDC-->>WS: service ticket
    WS->>Svc: HTTP/2 + Authorization: Negotiate [SPNEGO]
    Svc-->>WS: 200 + identity = chrisfu (PROLE.ORG)
```

Today's only Kerberized service is **git.prole.org (Gitea SPNEGO)** —
that one is `HTTP/git.prole.org@PROLE.ORG`, so the cross-realm
mechanic isn't actually exercised in production yet. The cross-realm
trust exists for **future services that live in the cluster (KNOE.LOCAL)**
and want to accept workstation TGTs from PROLE.ORG without forcing the
operator to maintain two parallel user databases.

### 4.4 Operational notes

- The `kerberos_trust_setup.yml` playbook is **idempotent**.
  Re-running it rotates the password (matches the cluster Secret)
  and bumps the kvno on the Samba side. If the cluster Secret is
  rotated, re-run the playbook to sync.
- MIT side kvno must match Samba side kvno; tonight's debug session
  (2026-05-11) walked through what happens when they drift.
- **pg_tde Transparent Data Encryption** is in
  `shared_preload_libraries` on the in-cluster Postgres. If you ever
  patch the cluster spec to remove it, the next pod restart will fail
  to open the encrypted data directory. Don't.

## 5. Gitea SPNEGO at `git.prole.org`

Shipped 2026-05-09. Workflow:

```mermaid
sequenceDiagram
    autonumber
    participant Br as Browser<br/>(PROLE.ORG-joined, Kerberos TGT)
    participant Trf as k3s Traefik
    participant SpnegoProxy as gitea-spnego-proxy
    participant Gitea as Gitea

    Br->>Trf: GET https://git.prole.org/
    Trf->>SpnegoProxy: pass-through
    SpnegoProxy-->>Br: 401 + WWW-Authenticate: Negotiate
    Br->>SambaKDC: get HTTP/git.prole.org service ticket
    Br->>SpnegoProxy: GET / + Authorization: Negotiate [blob]
    SpnegoProxy->>SpnegoProxy: validate SPNEGO with keytab<br/>(HTTP/git.prole.org@PROLE.ORG)
    SpnegoProxy->>Gitea: GET / + X-WEBAUTH-USER: chrisfu
    Note over Gitea: ENABLE_REVERSE_PROXY_AUTHENTICATION=true<br/>auto-provisions user on first sign-in
    Gitea-->>Br: 200 (signed-in as chrisfu)
```

Components:

- `gitea-spnego-proxy` Deployment (Go service, port 4000) with a
  `HTTP/git.prole.org@PROLE.ORG` keytab mounted from a k8s Secret
- Gitea env `ENABLE_REVERSE_PROXY_AUTHENTICATION=true` +
  `REVERSE_PROXY_AUTHENTICATION_HEADER=X-WEBAUTH-USER`
- Traefik IngressRoute routing `git.prole.org` → proxy → Gitea
- Gitea Helm-deployed; PVC on local-path SC; values managed in
  `gitea/deploy.sh`

For password fallback (1Password-stored), each engineer has a Gitea
recovery password set via `etc/init_knoe_users.sh:gitea_api_set_password`
(landed today after several false-positive-success rounds — see commits
`5d545d2`, `d5f6e8f`, `4f23b63`).

## 6. Supabase Studio web SSO

Two endpoints, two states.

### 6.1 `db.prole.org` (k3s, current)

Live since 2026-04-30. **Google OAuth** via `oauth2-proxy` with
`--email-domain=knoey.com`. Backed by the k3s Supabase deployment.
Auth flow is the standard oauth2-proxy → Kong → Studio chain
documented in the canonical doc.

Future plan: keep `db.prole.org` on Google OAuth as the homelab path
(prole workstations don't have a domain-joined identity that maps
to knoey.com naturally). A separate Kerberos-gated path can be
layered in later if needed.

### 6.2 `db.knoe.dev` (GKE, planned)

Brief: [`docs/plans/junie/per-user-supabase-workspace.md`](../docs/plans/junie/per-user-supabase-workspace.md).
Target state — swap oauth2-proxy + GoTrue from Google OAuth to
**knoe-auth OIDC** as the upstream, so engineers sign in once
(via knoe-auth) and land in their per-user Postgres schema.

**Blocked on:** [`pg_oauth-image-install.md`](https://github.com/dredx/knoe-db/blob/main/docs/plans/junie/pg_oauth-image-install.md)
— the knoe-db image doesn't yet contain the pg_oauth extension, so
the Studio → Postgres step can't validate JWTs.

## 7. Engineer psql access

Two paths active.

### 7.1 SCRAM (Phase 1; current default)

Per-engineer LOGIN role with a strong password rotated through
1Password (`knoey` vault). Works today, scales poorly. See
[`db-access.md`](db-access.md) §Phase 1.

### 7.2 OAUTHBEARER via Device Flow (Phase 2; shipped, blocked)

Brief: [`pg-oauth-resume.md`](plans/junie/pg-oauth-resume.md). Shipped
to source (commit `eef729a`) on 2026-05-11 with full spec changes to
`deploy/gcp/gke/knoe-db.yaml` and `etc/init_cnpg_gke.sh`, but the
matching pg_oauth extension was never installed in the knoe-db
Docker image. The cluster crashed on rollout attempt
(2026-05-11 evening); rolled back via `kubectl patch` to remove
`pg_oauth` from `shared_preload_libraries`. **Live cluster spec
currently diverged from git** — will reconcile when
[`pg_oauth-image-install.md`](https://github.com/dredx/knoe-db/blob/main/docs/plans/junie/pg_oauth-image-install.md)
lands.

## 8. Engineer onboarding (prole-specific steps)

1. **knoey.com Google Workspace** — invite `<short>@knoey.com`
   (existing operator with admin role does this in Google Admin).
2. **Samba AD** — provision the workstation account on myrddin:
   ```bash
   sudo samba-tool user create <short> <password>
   sudo samba-tool group addmembers prole-engineers <short>
   ```
3. **knoe-auth user table** — `etc/init_knoe_users.sh provision_user
   <primary> <short> <short>@knoey.com '<Full Name>' developer` (or
   use the lower-level `etc/onboard_engineer.sh`).
4. **Postgres role** — same per-engineer SCRAM role flow as
   [`db-access.md`](db-access.md) Phase 1. After
   `init_user_schemas_gke.sh` (or local k3s equivalent) lands the
   per-user schema brief, also add to the allowlist there.
5. **1Password access** — share the `knoey` vault.
6. **Workstation join (optional)** — if they'll log into a homelab
   workstation, run `realm join PROLE.ORG` plus the Ansible
   `workstation_kerberos.yml` playbook so their `/etc/krb5.conf` and
   browser policies are set up for SPNEGO.

## 9. Known fragility / current incidents

| Issue | First seen | State |
|---|---|---|
| pg_oauth extension missing from knoe-db image | 2026-05-11 | Open. Cluster spec rolled back; awaiting [`pg_oauth-image-install.md`](https://github.com/dredx/knoe-db/blob/main/docs/plans/junie/pg_oauth-image-install.md). |
| `init_user_schemas_gke.sh` silent no-op | 2026-05-11 | Open. Manually patched chrisfu + ron tonight; brief filed at [`init-user-schemas-heredoc-fix.md`](https://github.com/dredx/knoe-db/blob/main/docs/plans/junie/init-user-schemas-heredoc-fix.md). |
| `deploy.sh` calls non-existent `knoe.deploy_pipeline` module | Since rebrand 2026 commit `55b6a6a` | Open. Brief: [`deploy-pipeline-restore.md`](https://github.com/dredx/knoe-db/blob/main/docs/plans/junie/deploy-pipeline-restore.md). |
| MagicMock leaking into `conf/*.cfg` despite shipped fix | 2026-05-11 (after `a220400`) | Open. Original brief `todo-1-cfg-save-path-bug.md` is shipped but doesn't catch this shape; needs reopen. |
| KDC cross-realm trust needs a re-run after cluster Secret rotation | Always | Documented; idempotent playbook handles it. |
| Gitea admin-promotion PATCH resets password on SPNEGO users | 2026-05-10 | Fixed (commits `5d545d2`, `d5f6e8f`, `4f23b63`). |

## 10. Reference

| File / dir | Role |
|---|---|
| `infrastructure/playbooks/kerberos_trust_setup.yml` | Cross-realm trust on the Samba side |
| `infrastructure/playbooks/workstation_kerberos.yml` | Workstation `/etc/krb5.conf` + browser policy |
| `etc/init_knoe_users.sh` | User-table + Gitea + Kerberos principal provisioning |
| `etc/init_kdc.sh` | In-cluster MIT KDC + cross-realm krbtgts |
| `etc/init_1password.sh` | 1Password preflight + `knoey` vault bootstrap |
| `gitea/deploy.sh` | Gitea Helm install + SPNEGO proxy + reverse-proxy auth env |
| `k8s/knoe/gitea-spnego-proxy.yaml` | gitea-spnego-proxy Deployment + keytab Secret mount |
| `deploy/opentofu/k3s/manifests/knoe/` | k3s manifests (mirror of knoe-db's `deploy/gcp/gke/` + `k8s/knoe/`) |
| `~/.vault_pass` | Ansible vault password file (operator-local, not in repo) |
| `inventory/group_vars/ad_dc/vault.yml` | Samba admin password (Ansible vault) |
| `inventory/group_vars/all/vault_k3s.yml` | k3s join token + secrets |

## 11. See also

- [`~/dev/knoe-db/docs/knoe-auth.md`](https://github.com/dredx/knoe-db/blob/main/docs/knoe-auth.md) — canonical architecture
- [`db-access.md`](db-access.md) — engineer psql access (Phase 1 + 2)
- [`knoe-auth-phase-2.md`](knoe-auth-phase-2.md) — OIDC issuer Phase 2 design
- [`local-dev-knoe-auth.md`](local-dev-knoe-auth.md) — k3d dev loop for the knoe-auth pod
- [`oidc-setup.md`](oidc-setup.md) — Google OAuth client registration for oauth2-proxy + Grafana