prole/docs/plans/prole-auth-samba-ad-cross-realm.md
chrisfu cf33342500 feat(prole): bootstrap knoe-auth on k3s; tenant onboarding; cluster stabilisation
knoe-auth (prole.org k3s):
- Fix CNPG manifest drift: remove spec.backup.pluginConfiguration (CNPG 1.28 only),
  switch spec.certificates from serverTLSSecret to serverAltDNSNames
- Apply knoe-auth Round 1 schema + GRANTs manually (postInitSQL had never run on live cluster)
- Fix OIDC signing key generator: base64(DER) not base64(PEM) — OidcTokenService
  does Base64.decode() → PKCS8EncodedKeySpec which requires raw DER bytes
- Add OIDC controllers: authorize, token, userinfo, jwks, discovery
- Add prole Spring profile: cookieDomain, emailDomain, Kerberos config
- Add secret example templates: knoe-db-user, knoe-auth-oidc-signing, knoe-auth-google-prole
- Kong configmap: scope knoe-auth route to /auth prefix only

Tenant onboarding:
- Add etc/onboard_tenant.sh: provision/apply/rotate/status workflow backed by 1Password
  vaults; types: 'enterprise' (own Kerberos + domain) and 'tenant' (hosted, initContainer KDC)
- Provision 'Knoe Tenant - prole.org' vault; apply all 7 k8s secrets to knoe-system
- init_knoe_auth.sh: add explicit GRANT + ALTER DEFAULT PRIVILEGES for knoe role

Cluster stabilisation:
- gitea: roll back 14-day stuck rollout (RWO PVC + maxSurge=100% deadlock);
  patch deployment strategy to Recreate
- supabase: create supabase_admin role, _supabase db, _analytics schema, _realtime schema
  in CNPG — analytics and realtime had never connected since Helm install day 1
- knoe-db barman ObjectStore: add GCS-backed objectstore manifest + scheduled backup

Infrastructure:
- gandalf host_vars: k3s registry config
- pi host_vars: clean up stale entries
- knoe-db schemas: ekosystem.sql, ekosystem_objects.sql
- init_prole_app.sql: prole app DB initialisation

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-26 00:50:37 -07:00

5.9 KiB

prole.org auth — Samba AD cross-realm trust & keytab provisioning

Status: Infrastructure coded; operational steps pending. Owner: chrisfu

Closes Path B: Samba AD users on myrddin.prole.org → Kerberos SPNEGO → knoe-auth session.


Architecture recap

The in-cluster KDC runs the KNOE.LOCAL realm. myrddin.prole.org (10.0.0.3) runs Samba AD with realm PROLE.ORG. A bidirectional cross-realm trust lets a PROLE.ORG ticket-holder authenticate to any kerberized in-cluster service (HTTP/api.prole.org@KNOE.LOCAL etc.) without needing a second Kerberos account.

AD user on myrddin   →   TGT from PROLE.ORG KDC (myrddin:88)
                     →   cross-realm referral → KNOE.LOCAL KDC (in-cluster)
                     →   service ticket for HTTP/api.prole.org@KNOE.LOCAL
                     →   SPNEGO negotiation with knoe-auth
                     →   knoe_session cookie issued

The in-cluster ExternalName service prole-kerberos-ad-dc.knoe-system.svc.cluster.local:88 routes to myrddin.prole.org so the KDC container can find the PROLE.ORG KDC.


Step 1 — Create the inter-realm keys on the in-cluster KDC

init_kdc.sh provisions the MIT KDC side when PROLE_KDC_TRUST_REALM is set. Run this from within the cluster (or via kubectl exec into the KDC container):

# The shared trust password is in knoe-kdc-secrets/trust_shared_password
TRUST_SHARED_PW=$(kubectl -n knoe-system get secret knoe-kdc-secrets \
  -o jsonpath='{.data.trust_shared_password}' | base64 -d)

kubectl exec -n knoe-system deploy/knoe-auth -c kdc -- kadmin.local -q \
  "addprinc -pw ${TRUST_SHARED_PW} krbtgt/PROLE.ORG@KNOE.LOCAL"

kubectl exec -n knoe-system deploy/knoe-auth -c kdc -- kadmin.local -q \
  "addprinc -pw ${TRUST_SHARED_PW} krbtgt/KNOE.LOCAL@PROLE.ORG"

Both directions must exist. The KNOE.LOCAL → PROLE.ORG key is used when a PROLE.ORG principal requests a service ticket in KNOE.LOCAL (referral chain).


Step 2 — Create the reciprocal trust account on myrddin (Samba side)

SSH to myrddin.prole.org as Administrator and run:

# Create the outbound trust principal that KNOE.LOCAL will use
sudo samba-tool user create krbtgt_KNOEDOTLOCAL --random-password

# Set the inter-realm key to the SAME shared password used in Step 1
sudo samba-tool user setpassword krbtgt_KNOEDOTLOCAL --newpassword="${TRUST_SHARED_PW}"

# Disable password expiry for the trust account
sudo samba-tool user setexpiry krbtgt_KNOEDOTLOCAL --noexpiry

# Create the one-way trust entry (PROLE.ORG trusts KNOE.LOCAL)
sudo samba-tool domain trust create KNOE.LOCAL \
  --type=external \
  --direction=incoming \
  --password="${TRUST_SHARED_PW}"

Note: KNOE.LOCAL must be resolvable from myrddin. Either add a DNS forwarder for the KNOE.LOCAL domain pointing at the in-cluster KDC service IP, or add a hosts entry.


Step 3 — Extract the HTTP service keytab

The knoe-auth pod needs HTTP/api.prole.org@PROLE.ORG to accept SPNEGO from PROLE.ORG browsers.

# Option A — generate keytab on myrddin (if the principal lives in PROLE.ORG)
ssh myrddin.prole.org "sudo samba-tool user create HTTP-api-prole-org --random-password && \
  sudo samba-tool spn add HTTP/api.prole.org HTTP-api-prole-org && \
  sudo samba-tool domain exportkeytab /tmp/http-api.keytab --principal=HTTP/api.prole.org"
scp myrddin.prole.org:/tmp/http-api.keytab ./http.keytab

# Option B — generate on the in-cluster KDC (principal in KNOE.LOCAL)
kubectl exec -n knoe-system deploy/knoe-auth -c kdc -- kadmin.local -q \
  "addprinc -randkey HTTP/api.prole.org@KNOE.LOCAL"
kubectl exec -n knoe-system deploy/knoe-auth -c kdc -- kadmin.local -q \
  "ktadd -k /tmp/http.keytab HTTP/api.prole.org@KNOE.LOCAL"
kubectl cp knoe-system/$(kubectl get pod -n knoe-system -l app=knoe-auth -o name | head -1 | cut -d/ -f2):/tmp/http.keytab ./http.keytab

Choose Option A if clients authenticate as user@PROLE.ORG — the service principal must match the realm clients target. Use Option B if you want KNOE.LOCAL to be authoritative.

Store the keytab in the k8s secret:

kubectl create secret generic knoe-auth-http-keytab \
  --namespace knoe-system \
  --from-file=http.keytab=./http.keytab \
  --dry-run=client -o yaml | kubectl apply -f -

The deployment mounts this at /etc/knoe-auth/http.keytab (already wired in knoe-auth-deployment.yaml).


Step 4 — Verify trust end-to-end

From a domain-joined Windows or Linux machine in PROLE.ORG:

# Linux (kinit from PROLE.ORG)
kinit user@PROLE.ORG
kvno HTTP/api.prole.org@KNOE.LOCAL   # should succeed via cross-realm referral

# Test SPNEGO login
curl -v --negotiate -u : https://api.prole.org/auth/spnego
# Expect: 302 redirect with knoe_session cookie

Step 5 — AD group sync (future work)

AD group membership → knoe.access_grant rows is not yet implemented. Current behavior:

  • Samba AD users get a baseline knoe_session with no extra groups.
  • Admin rights are granted only to usernames listed in PROLE_AUTH_ADMIN_PRINCIPALS.

To grant admin access to an AD user before group sync is built:

kubectl -n knoe-system set env deploy/knoe-auth \
  PROLE_AUTH_ADMIN_PRINCIPALS="admin,<samba-username>"

File Purpose
deploy/opentofu/k3s/manifests/knoe/prole-kerberos-ad-dc-svc.yaml ExternalName service → myrddin.prole.org:88
deploy/opentofu/k3s/manifests/knoe/knoe-kdc-secrets.example.yaml trust_shared_password lives here
etc/init_kdc.sh Automates Step 1 when PROLE_KDC_TRUST_* env vars are set
deploy/opentofu/k3s/manifests/knoe/knoe-auth-http-keytab-secret.example.yaml HTTP service keytab secret template