mirror of
https://github.com/dredx/prole.git
synced 2026-09-23 10:13:58 +00:00
docs(runbook): stage db.prole.org GSSAPI auth for the airgap lane
Complete, review-ready runbook to light up Kerberos/GSSAPI on db.prole.org so `kn db --service prole.org` authenticates with a local PROLE.ORG ticket — the airgap data lane (no password, no knoe-auth, no internet). Nothing applied. Delivers: - Samba SPN + keytab export steps (myrddin): postgres/db.prole.org@PROLE.ORG - k8s keytab secret (knoe-db-gss-keytab-secret.example.yaml template) - exact knoe-db.yaml diff: krb_server_keyfile, hostgssenc pg_hba (include_realm=0 → role), projectedVolumeTemplate keytab mount, chrisfu/ron managed roles - apply + end-to-end verify (kinit → kn db --service prole.org) - failure-mode triage, rollback, follow-ups Hooks into the manifest's existing placeholders (knoe-db.yaml lines 38 + 45, which already note 'Kerberos disabled on k3s'). Image is already --with-gssapi. Flags the PROLE.ORG (Kerberos) vs PROLE.LOCAL (knoe.user default) discrepancy to reconcile. Grants intentionally deferred — connect succeeds, reads gated. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
chrisfu
parent
76522eace2
commit
c9591a5da2
@ -0,0 +1,26 @@
|
||||
# knoe-db-gss-keytab — Kerberos service keytab for db.prole.org GSSAPI auth.
|
||||
#
|
||||
# Holds the keytab for the Postgres service principal so libpq GSSAPI clients
|
||||
# (e.g. `kn db --service prole.org`) can authenticate with a local PROLE.ORG
|
||||
# ticket — no password, fully airgap-capable.
|
||||
#
|
||||
# DO NOT commit the real keytab. This is a template; create the live secret from
|
||||
# the keytab exported on myrddin (see docs/runbooks/db-prole-gssapi.md):
|
||||
#
|
||||
# sudo samba-tool domain exportkeytab /tmp/pg.keytab \
|
||||
# --principal=postgres/db.prole.org@PROLE.ORG
|
||||
# kubectl --context=prole-service-cluster -n knoe create secret generic \
|
||||
# knoe-db-gss-keytab --from-file=postgres.keytab=/tmp/pg.keytab
|
||||
# rm -f /tmp/pg.keytab # shred the keytab off disk
|
||||
#
|
||||
# The CNPG cluster mounts this via spec.projectedVolumeTemplate at
|
||||
# /projected/postgres.keytab and points krb_server_keyfile there.
|
||||
apiVersion: v1
|
||||
kind: Secret
|
||||
metadata:
|
||||
name: knoe-db-gss-keytab
|
||||
namespace: knoe
|
||||
type: Opaque
|
||||
data:
|
||||
# base64 of the binary keytab for postgres/db.prole.org@PROLE.ORG
|
||||
postgres.keytab: <BASE64_KEYTAB_HERE>
|
||||
175
docs/runbooks/db-prole-gssapi.md
Normal file
175
docs/runbooks/db-prole-gssapi.md
Normal file
@ -0,0 +1,175 @@
|
||||
# Runbook — GSSAPI (Kerberos) auth on db.prole.org
|
||||
|
||||
**Status:** staged for review — nothing here is applied yet.
|
||||
**Goal:** let `kn db --service prole.org` authenticate to `db.prole.org` with a
|
||||
local PROLE.ORG Kerberos ticket via libpq GSSAPI. No password, no knoe-auth, no
|
||||
outbound network — the airgap data lane.
|
||||
|
||||
## Why GSSAPI (not OAUTHBEARER) for the airgap lane
|
||||
|
||||
The cloud lane (`db.0.knoe.dev`) uses OAUTHBEARER + pg_knoe_auth against
|
||||
knoe-auth's JWKS — that needs network to the issuer. The airgap lane's KDC
|
||||
(`myrddin.prole.org`, realm `PROLE.ORG`) is **on the LAN**, and the workstation
|
||||
already holds a `PROLE.ORG` TGT. libpq's native GSSAPI uses that ticket
|
||||
directly, so the DB lane works with zero internet. `kn db` already selects this
|
||||
lane for `--service prole.org` (commit `a06ddad`); this runbook lights up the
|
||||
server side.
|
||||
|
||||
## Preconditions (verify first)
|
||||
|
||||
- **Realm:** workstation `default_realm = PROLE.ORG`, `kdc = myrddin.prole.org`
|
||||
(confirmed in `/etc/krb5.conf`). Note: `knoe-db.yaml` `knoe.user.realm`
|
||||
defaults to `PROLE.LOCAL` — that's a stale app default, **not** the Kerberos
|
||||
realm. Use `PROLE.ORG` throughout. (Worth reconciling separately.)
|
||||
- **Image:** `registry.prole.org/knoe-system/knoe-db:18-*` is built
|
||||
`--with-gssapi` (confirmed via `pg_config --configure`). No image change.
|
||||
- **DNS:** `db.prole.org` resolves on the LAN to the k3s LB (10.0.0.x). The
|
||||
Postgres SPN host (`postgres/db.prole.org`) must match what clients connect to.
|
||||
|
||||
---
|
||||
|
||||
## Step 1 — Samba: create the service principal + export the keytab (on myrddin)
|
||||
|
||||
```bash
|
||||
ssh myrddin.prole.org
|
||||
# A dedicated service account holds the SPN (cleaner than the machine account):
|
||||
sudo samba-tool user create knoe-db-pg --random-password \
|
||||
--description="Postgres GSSAPI service principal for db.prole.org"
|
||||
sudo samba-tool spn add postgres/db.prole.org knoe-db-pg
|
||||
|
||||
# Export a keytab for exactly that principal:
|
||||
sudo samba-tool domain exportkeytab /tmp/pg.keytab \
|
||||
--principal=postgres/db.prole.org@PROLE.ORG
|
||||
sudo chown "$USER" /tmp/pg.keytab
|
||||
|
||||
# Sanity: the keytab should list postgres/db.prole.org@PROLE.ORG
|
||||
klist -k /tmp/pg.keytab
|
||||
```
|
||||
|
||||
Copy `/tmp/pg.keytab` to where you run `kubectl`, then **shred it on myrddin**:
|
||||
`shred -u /tmp/pg.keytab`.
|
||||
|
||||
## Step 2 — Create the k8s keytab secret
|
||||
|
||||
```bash
|
||||
kubectl --context=prole-service-cluster -n knoe create secret generic \
|
||||
knoe-db-gss-keytab --from-file=postgres.keytab=/tmp/pg.keytab
|
||||
shred -u /tmp/pg.keytab # remove the keytab from disk
|
||||
```
|
||||
|
||||
(Template: `manifests/knoe/knoe-db-gss-keytab-secret.example.yaml`.)
|
||||
|
||||
## Step 3 — Patch the CNPG cluster (`manifests/knoe/knoe-db.yaml`)
|
||||
|
||||
Three edits. **Diff:**
|
||||
|
||||
```yaml
|
||||
postgresql:
|
||||
parameters:
|
||||
shared_buffers: 256MB
|
||||
pg_stat_statements.max: '10000'
|
||||
pg_stat_statements.track: all
|
||||
- # krb_server_keyfile omitted — Kerberos disabled on k3s (no keytab secret)
|
||||
+ # Kerberos: keytab mounted via projectedVolumeTemplate at /projected
|
||||
+ krb_server_keyfile: '/projected/postgres.keytab'
|
||||
shared_preload_libraries:
|
||||
- pg_stat_statements
|
||||
- pg_tde
|
||||
pg_hba:
|
||||
- local all postgres trust
|
||||
- local all knoe scram-sha-256
|
||||
- # SCRAM auth — Kerberos/GSS rules omitted (kerberos_enabled=false on k3s)
|
||||
+ # GSSAPI: PROLE.ORG ticket → role (include_realm=0 strips @PROLE.ORG).
|
||||
+ # Placed before the catch-all scram rules so gss is tried first.
|
||||
+ - hostgssenc knoe-db chrisfu 0.0.0.0/0 gss include_realm=0 krb_realm=PROLE.ORG
|
||||
+ - hostgssenc knoe-db ron 0.0.0.0/0 gss include_realm=0 krb_realm=PROLE.ORG
|
||||
- host all postgres all scram-sha-256
|
||||
- host knoe knoe-db all scram-sha-256
|
||||
- host all all all scram-sha-256
|
||||
- hostssl knoe knoe-db all scram-sha-256
|
||||
```
|
||||
|
||||
Mount the keytab (CNPG `projectedVolumeTemplate` lands sources under `/projected`):
|
||||
|
||||
```yaml
|
||||
spec:
|
||||
# … existing fields …
|
||||
projectedVolumeTemplate:
|
||||
sources:
|
||||
- secret:
|
||||
name: knoe-db-gss-keytab
|
||||
items:
|
||||
- key: postgres.keytab
|
||||
path: postgres.keytab
|
||||
```
|
||||
|
||||
Add the login roles (under `managed.roles`, mirroring `admin`):
|
||||
|
||||
```yaml
|
||||
managed:
|
||||
roles:
|
||||
- name: chrisfu
|
||||
ensure: present
|
||||
login: true
|
||||
comment: "Kerberos GSSAPI identity (PROLE.ORG)"
|
||||
- name: ron
|
||||
ensure: present
|
||||
login: true
|
||||
comment: "Kerberos GSSAPI identity (PROLE.ORG)"
|
||||
```
|
||||
|
||||
> Grants are deliberately omitted — these roles can log in but read nothing
|
||||
> until granted. Decide schema access separately (mirrors the cloud lane, where
|
||||
> a connect succeeds but `permission denied for schema …` until granted).
|
||||
|
||||
## Step 4 — Apply
|
||||
|
||||
```bash
|
||||
# Whichever applies this cluster (ArgoCD app-of-apps, or direct):
|
||||
kubectl --context=prole-service-cluster apply \
|
||||
-f deploy/opentofu/k3s/manifests/knoe/knoe-db.yaml
|
||||
# CNPG reconciles pg_hba + the projected keytab without a full restart;
|
||||
# a config reload is enough. Watch:
|
||||
kubectl --context=prole-service-cluster -n knoe get cluster knoe-db -w
|
||||
```
|
||||
|
||||
## Step 5 — Verify the airgap lane end to end
|
||||
|
||||
```bash
|
||||
kinit chrisfu@PROLE.ORG # local TGT, no internet
|
||||
kn db --service prole.org -c "select current_user, session_user"
|
||||
# → chrisfu chrisfu (GSSAPI, no password)
|
||||
```
|
||||
|
||||
If it fails, check the Postgres log for the GSSAPI reason:
|
||||
```bash
|
||||
POD=$(kubectl --context=prole-service-cluster -n knoe get pods \
|
||||
-l cnpg.io/instanceRole=primary -o jsonpath='{.items[0].metadata.name}')
|
||||
kubectl --context=prole-service-cluster -n knoe logs "$POD" -c postgres --tail=40 \
|
||||
| grep -iE "gss|krb|authentication"
|
||||
```
|
||||
|
||||
Common causes:
|
||||
- *"GSSAPI ... no credentials"* → keytab principal ≠ `postgres/db.prole.org@PROLE.ORG`
|
||||
(re-export, confirm `klist -k`).
|
||||
- *server falls back to scram ("no password supplied")* → the `hostgssenc` rule
|
||||
didn't match (db/user/order), or `gssencmode` not negotiated.
|
||||
- *clock skew* → Kerberos needs workstation and KDC within ~5 min; check NTP.
|
||||
|
||||
---
|
||||
|
||||
## Rollback
|
||||
|
||||
Remove the two `hostgssenc` lines and `krb_server_keyfile`, re-apply. The secret
|
||||
and roles are inert without the pg_hba rules. Delete the secret/roles if desired.
|
||||
|
||||
## Notes / follow-ups
|
||||
|
||||
- **PROLE.ORG vs PROLE.LOCAL** in `knoe.user.realm` — reconcile so the app's
|
||||
notion of realm matches Kerberos.
|
||||
- **Grants** — define what `chrisfu`/`ron` (and future group roles) may read.
|
||||
The cloud lane carries a `groups` claim; the airgap lane can map AD group
|
||||
membership to roles for parity.
|
||||
- **Same source of truth** — once this lands, an airgap project + its local MCP
|
||||
both point at `db.prole.org`, exactly as a cloud project + MCP share
|
||||
`db.0.knoe.dev`.
|
||||
Loading…
Reference in New Issue
Block a user