Captured for the knoe-db convergence (Phase 2): configure_kubernetes_auth() (OpenBao
K8s auth → scoped knoe-jobs tokens vs root), the Kong knoe-secret route (secure-dropbox
bridge /secret/* → knoe-jobs:8081), and the prole.org canary-deploy runbook. None of
this is in canonical knoe-db — it is prole-staging-specific, not stale.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Add infrastructure/logs/ to .gitignore so ansible run logs
(infrastructure/logs/ansible/*.log) are never accidentally committed
- Track docs/plans/junie/ekosystem-uuid-cnpg-wire.md — Junie brief for
wiring the ekosystem UUID schema to the CNPG cluster (queue item #13)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
docs/completed/ — new directory; 9 shipped Junie briefs moved from
docs/plans/junie/ (02-k3s-prole-rename, 03-image-rename, 06-patch-garage,
07-init-cnpg-gke-sa-wiring, 13-podmonitor, 15-remove-dead-dashboard,
k3d-knoe-auth-dev-loop, phase2-oidc-gke-deploy, todo-1-cfg-save-path-bug).
docs/completed/README.md summarises all shipped work with dates/refs.
docs/plans/junie/README.md — updated to 2026-05-23; active/pending tables
reflect current state; shipped section now points to docs/completed/.
conf/service/knoe.cfg — new unattended deploy config for the service/k3s
environment (used by ./deploy.sh).
Co-authored-by: Junie <junie@jetbrains.com>
Mermaid 10.7.0 chokes on bare <placeholder> in message text, parsing
it as HTML. Swap to square brackets in the cross-realm §4.3 sequence
diagram and the Gitea SPNEGO §5 diagram:
<svc-host>, <SPNEGO> (§4.3)
<blob> (§5)
Matches the same fix landed in knoe-db's docs/knoe-auth.md.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Companion to the canonical architecture doc at
~/dev/knoe-db/docs/knoe-auth.md. Documents what's specific to the
prole.org homelab deployment:
- Two upstream identity layers (Samba AD on myrddin for PROLE.ORG
Kerberos, Google Workspace knoey.com for OIDC)
- Three deploy targets (k3s on myrddin/merlin/gandalf, k3d for
engineer dev loop, GKE for production API endpoint)
- Hostname map (myrddin/morgana/zinfandel/db.prole.org/
git.prole.org/api.knoe.dev/db.knoe.dev/etc.)
- Cross-realm Kerberos trust setup (KNOE.LOCAL ↔ PROLE.ORG) via
infrastructure/playbooks/kerberos_trust_setup.yml — RC4 pin
explanation, kvno sync notes
- Gitea SPNEGO at git.prole.org with Mermaid flow diagram
- Supabase Studio status (db.prole.org Google OAuth working;
db.knoe.dev knoe-auth OIDC blocked on pg_oauth image)
- Engineer onboarding checklist (prole-specific steps)
- Known fragility / current incidents table
Mermaid sequence diagram covers the cross-realm flow end-to-end
(workstation → Samba KDC → MIT KDC → service ticket), distinct from
the canonical doc's generic version.
References the matching Junie briefs in knoe-db for the open
fragility items (pg_oauth-image-install, init-user-schemas-heredoc-fix,
deploy-pipeline-restore).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Mirrors the knoe-db commit `ff7546d` patches into the prole copy of
`etc/init_kdc.sh` so a re-run of `install.sh --mode k3s --reset` from
this repo produces a working cross-realm trust without manual cluster
surgery. The k3s cluster is provisioned from this repo, so the source
fix must live here (knoe-db remains canonical for GKE).
Changes to etc/init_kdc.sh:
1. Create BOTH cross-realm krbtgts in MIT, not just the outbound one.
The inbound `krbtgt/<REALM>@<TRUST_REALM>` (issued by Samba,
decrypted here) was missing entirely; without it, MIT cannot
decrypt inbound TGTs and the trust never carries traffic.
2. Pin both cross-realm krbtgts to RC4 (`arcfour-hmac:normal`). AES
keys depend on salt, and Samba's `<remote_realm>+UPN` salt does
not match MIT's `<local_realm>+<principal-no-realm>`; RC4 has no
salt so both sides converge from the password alone. Matches the
already-pinned Samba side (commit `ad1eced`).
3. Replace the broken "remote kadmin to Samba" reciprocal-trust block
with a documented no-op pointing at
`infrastructure/playbooks/kerberos_trust_setup.yml`. Samba AD
does not accept additions over MIT's kadmin protocol; the block
always failed with "Missing parameters in krb5.conf required for
kadmin client".
4. Switch the KDC data volume from emptyDir to a PVC
(claimName `knoe-kdc-data`, parameterized by
`$PROLE_KDC_STORAGE_SIZE` and `$PROLE_KDC_STORAGE_CLASS`).
State now survives pod restarts.
Adds Junie brief `docs/plans/junie/kdc-trust-reset-repeatable.md`
with four TDD acceptance criteria for an end-to-end --reset run on
the prole k3s cluster.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Followup from commit dba8a2d's findings. ConfigMixin._save_knoe_cfg in
knoe/ui/screens/cfg.py reads Tk widget vars via `var.get()` and writes
the result to conf/<mode>.cfg. When a var is a MagicMock (interactive
./install.py run in a non-Tk context, partially-mocked widgets), the
save path serializes the mock's repr-string into the cfg, then the next
installer pass calls os.makedirs() on those values and produces
directories literally named `<MagicMock name='Canvas().tk.call().strip()'
id='4743999712'>/`.
The brief lays out a TDD approach for Junie:
1. Write failing test at tests/installer/test_cfg_save_refuses_mock_values.py
that passes MagicMock widget vars and asserts _save_knoe_cfg raises
TypeError naming the field.
2. Implement the minimal fix: a `_str_value(var, field=...)` helper in
cfg.py that validates widget reads and raises if non-str. Use it
in the .get()/.strip() chains across lines 103-155.
3. Verify the 750 existing installer tests still pass.
Brief includes file pointers (cfg.py:44 _save_knoe_cfg, line 102
globals_to_save assembly, line 277 cfg_path.write_text), the canonical
failing test stub, both fix-approach options (per-read validator vs
end-of-flow dict walk), and explicit commit-shape guidance.
Index updates:
docs/plans/junie/README.md — todo-1 row added under Active
docs/TODO.md §"In progress" — todo-1 promoted above Phase 3 (smaller
scope, easy to land first)
Naming convention: `todo-N-<slug>.md` for follow-up bugs, distinct from
the `NN-<slug>.md` pattern reserved for ranked queue items.
Phase 1 (host loop) and the Phase 2 OIDC sandbox are shipped; this
brief queues Phase 3 of k3d-mirror-of-GKE: build the knoe-auth image,
k3d-import, run as a pod inside the cluster. Pre-merge smoke loop —
not the daily inner loop.
Deliverables (see brief for the full Definition of done):
- k8s/knoe/knoe-auth-deployment.yaml (NEW; sibling of GKE manifest)
- scripts/k3d-knoe-deploy.sh (NEW; build + import + apply + wait)
- Makefile k3d-knoe-{deploy,redeploy,undeploy} targets
- Phase 2 signing key flows from etc/secrets/knoe-auth-oidc-key.b64 into
a knoe-auth-oidc-signing-key K8s Secret in knoe-system, mirroring how
the GKE deploy reads it.
Index updates:
docs/plans/junie/README.md — k3d Phase 3 in Active row
docs/TODO.md §"In progress" — promotes the brief to top
docs/plans/k3d-gke-mirror.md — banner + §6 Phase 3 entry
flipped from "out of scope"
to "in flight"
Three independent lines of work landing in one commit because they were all
on disk together end-of-session and the cross-file edits (TODO, knoe-system,
junie/README) interleave cleanly:
1. **Queue #3 — image rename `knoe-authority` → `knoe-auth`** (Junie).
Closes drift R6. The Maven artifact has been `knoe-auth.jar` since
commit b355855; the deploy manifests now match.
authority/Dockerfile.app (NEW)
deploy/gcp/gke/knoe-auth-deployment.yaml (3 image tags renamed)
deploy/opentofu/k3s/manifests/knoe/knoe-auth-deployment.yaml (2 image tags renamed)
Makefile (docker-build-auth +
docker-push-auth + REGISTRY/KNOE_AUTH_VERSION
defaults)
docs/plans/junie/03-image-rename-knoe-authority-to-knoe-auth.md (brief, kept as design record)
2. **Phase 2 OIDC provider — GKE deploy** (Junie). Source landed via the
merge that brought claude/crazy-bose-fec256 back; the k3d sandbox
shipped earlier today (commit 93157b0). This commit completes the GKE
path: Kong route `/auth` → knoe-auth, default values added, signing
key wired via `knoe-auth-oidc` secret per the brief.
supabase/helm/knoe-supabase/templates/kong/config.yaml (Kong route /auth, gated
on .Values.knoeAuth.enabled)
supabase/helm/knoe-supabase/values.yaml (knoeAuth.enabled=false default;
studioIngress block — also
fixes the helm lint issue noted
in earlier commit 03bb731)
docs/plans/junie/phase2-oidc-gke-deploy.md (brief, kept as design record)
3. **k3d dev-user seed: `chrisfu` + `knoe_developer`** (Claude). Closes
"I want auth as chrisfu@knoey.com to my local cnpg database" for the
local dev loop. `etc/init_knoe_auth.sh --mode k3d` now runs
`seed_dev_users_k3d()` after the schema bootstrap, creating:
- `knoe_developer` group role with R/W on `knoe`+`public` (mirrors
the GKE production layout from docs/db-access.md; was hand-rolled
in production per the 2026-04-30 onboarding work, never baked into
postInitTemplateSQL).
- `chrisfu` LOGIN role with password `chrisfu-dev` (idempotent —
resets on every `make k3d-knoe-up`, so the rebuild loop is
deterministic). Granted into `knoe_developer`.
From the host with port-forward up:
PGPASSWORD=chrisfu-dev psql "postgresql://chrisfu@localhost:5432/knoe-db?sslmode=require"
Cross-cutting doc updates:
docs/TODO.md — Phase 2 GKE entry removed from "In progress"
(now "(none — all items shipped or paused)");
queue #3 + reality table R6 removed; pg_oauth
paused note flipped to "OIDC issuer now
deployed to GKE — pg_oauth can resume"; Done
section gets new entries for #3, Phase 2 GKE,
and chrisfu seed.
docs/knoe-system.md — Phase 2 GKE row → Shipped; pg_oauth → Ready;
knoe-auth-deployment.yaml note "knoe-auth:latest"
instead of "knoe-authority:latest"; "One-time
cluster setup" mentions the chrisfu seed.
docs/local-dev-knoe-auth.md — "Verify psql connectivity" rewritten to
connect as chrisfu (was `knoe`); new "Reset /
rebuild loop" section explaining idempotency.
docs/plans/junie/README.md — two new rows under Shipped: brief 03 and
phase2-oidc-gke-deploy.
Verification (the bits I ran locally):
- bash -n etc/init_knoe_auth.sh OK
- The seed SQL is idempotent (DO blocks with EXISTS checks +
ALTER ROLE on the password reset path)
- The Kong /auth route is gated on .Values.knoeAuth.enabled — default
false, no behavior change until someone flips it on per Junie's brief
Out of scope for this commit:
- Actually flipping knoeAuth.enabled=true on the live GKE chart and
rolling out — Junie's brief covers the runbook; needs the
knoe-auth-oidc K8s secret populated from 1Password first
- Rebuilding + pushing the new knoe-auth:latest image — `make
docker-push-auth` is wired, just hasn't been run yet
- Round 1.5 OpenBao transit-key encryption (still queued)
Co-authored-by: Junie <junie@jetbrains.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Completes the Phase 2 OIDC laptop dev path. Source for the OIDC surface
(discovery, authorize, token, userinfo, JWKS controllers + signing /
session services) landed on `main` via the merge that brought
claude/crazy-bose-fec256 back. This commit makes Phase 2 actually
exercisable in the k3d dev loop without any GKE deploy.
What's new:
authority/src/main/resources/application-k3d.yml
Spring profile activated by `-Dspring-boot.run.profiles=k3d`.
Enables OIDC (knoe.oidc.enabled=true), points the issuer at
`http://localhost:8080`, sets Kerberos realm to KNOE.LOCAL, and
aligns the datasource with the port-forwarded localhost:5432 DB.
etc/gen_oidc_signing_key.sh (executable)
Idempotent RS256 PKCS#8 keypair generator. Outputs:
etc/secrets/knoe-auth-oidc-key.pem (PEM)
etc/secrets/knoe-auth-oidc-key.b64 (single-line base64 of DER —
directly consumable as KNOE_AUTH_OIDC_SIGNING_KEY by
OidcTokenService.init())
`etc/secrets/` is already gitignored. Set FORCE=1 to rotate.
What's wired:
scripts/k3d-knoe-up.sh
New §7 calls etc/gen_oidc_signing_key.sh after schema seed. Keypair
persists across `make k3d-knoe-down && make k3d-knoe-up` cycles.
scripts/k3d-knoe-pf.sh
Output now includes the KNOE_AUTH_OIDC_SIGNING_KEY export line, the
full `mvn spring-boot:run` invocation with -Dspring-boot.run.profiles=k3d,
and the three OIDC endpoints to curl-test.
What's documented:
docs/local-dev-knoe-auth.md
"Daily loop" Terminal B: now exports KNOE_AUTH_OIDC_SIGNING_KEY,
runs with `-Dspring-boot.run.profiles=k3d`, and the verify section
includes /jwks.json. IntelliJ run config: adds Active Profiles: k3d
and a note about pasting the b64 directly (no shell expansion in
the env-var field).
docs/knoe-system.md
Phase 2 status row split: "k3d setup" → Shipped, "GKE deploy" →
Pending. The "Open work items" Phase 2 entry rewritten to flag
that the GKE deploy is the remaining thread (gated on queue #3 for
the image rebuild as `knoe-auth:latest`).
docs/TODO.md
Promoted "Phase 2 OIDC provider — GKE deploy" into §In progress
(replacing the empty "(none)" placeholder). Done section updated
with two entries: the k3d Phase 1 dev loop (Junie's c3...) and
this Phase 2 OIDC k3d sandbox.
End-to-end loop the engineer can run:
make k3d-knoe-up # one-time, ~5 min
make k3d-knoe-pf & # port-forwards
export KRB5_CONFIG=$PWD/etc/krb5.local.conf
export KNOE_AUTH_OIDC_SIGNING_KEY=$(cat etc/secrets/knoe-auth-oidc-key.b64)
mvn -pl authority spring-boot:run \
-Dspring-boot.run.jvmArguments="-Djava.security.krb5.conf=$PWD/etc/krb5.local.conf" \
-Dspring-boot.run.profiles=k3d
# then:
curl -s http://localhost:8080/.well-known/openid-configuration | jq .issuer
# → "http://localhost:8080"
curl -s http://localhost:8080/jwks.json | jq '.keys[0].kty'
# → "RSA"
Verified locally: keypair generator round-trips through openssl pkey -inform DER
(produces valid 2048-bit RSA keys); idempotent (existing key kept by default,
FORCE=1 rotates); bash -n clean on all 5 touched scripts.
Out of scope (TODO §In progress captures it):
- GKE deploy of Phase 2 (image rebuild + K8s Secret + deployment env vars)
- SPNEGO E2E from host browsers (k3d-mirror Phase 2)
- knoe-auth-as-pod in k3d (k3d-mirror Phase 3)
- OidcCodeService DB persistence (separate track)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Brings up the smallest k3d-resident stack that lets a host-side knoe-auth
(run via `mvn spring-boot:run` or IntelliJ) iterate against real Postgres
+ Kerberos. Closes Phase 1 of the k3d-gke-mirror plan (docs/plans/k3d-gke-mirror.md).
Scope:
- k8s/knoe/knoe-kdc-{configmap,deployment,service,pvc,init-job}.yaml
NEW; standalone KDC, realm KNOE.LOCAL (distinct from KNOE.DEV).
- etc/init_knoe_auth.sh: --mode k3d flag added; swaps realm + skips
GCP-specific steps. GKE behavior unchanged when flag absent.
- Makefile: k3d-knoe-up, k3d-knoe-pf, k3d-knoe-down (delegate to
scripts/k3d-knoe-{up,pf,down}.sh).
- scripts/k3d-knoe-{up,pf,down,smoke}.sh NEW; up = full bring-up,
pf = three port-forwards (5432/88/464) + JDBC URL + ^C cleanup,
down = teardown, smoke = sanity check.
- etc/krb5.local.conf NEW; checked-in libdefaults+realms config
pointing at localhost:88. udp_preference_limit=1 to dodge
kubectl port-forward UDP flakiness on macOS.
- docs/local-dev-knoe-auth.md NEW; one-time setup + daily loop +
IntelliJ run config.
- docs/knoe-system.md NEW; unified reference for the knoe-auth
service (GKE deployment + k3d dev loop + schema overview +
source map + open work items).
Verified per the brief's Definition of done: fresh-clone laptop can
`make k3d-knoe-up` + `make k3d-knoe-pf` + `mvn -pl authority spring-boot:run`
and hit /health, /.well-known/openid-configuration in <8 minutes.
Out of scope (parent plan docs/plans/k3d-gke-mirror.md §6):
- SPNEGO from host browsers (Phase 2)
- knoe-auth-as-pod / image build/load (Phase 3)
- Supabase stack on k3d (Phase 4)
- OidcCodeService DB persistence (separate track)
docs/plans/junie/README.md — k3d brief moved from Active to Shipped.
docs/TODO.md — In-progress now empty; Phase 2 pg_oauth notes that the
local dev loop is in place so it can resume.
Closes Phase 1; Phase 2+ briefs filed as needed.
Co-authored-by: Junie <junie@jetbrains.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The next development thread is knoe-auth Phase 2 OIDC iteration, which
just landed on main but has no fast inner-loop. Instead of bouncing
edits through the GKE deploy chain, we want a laptop-resident model
where knoe-auth runs from the IDE / mvn against real Postgres + KDC
brought up in k3d.
Two new docs:
docs/plans/k3d-gke-mirror.md
Architectural plan. Top-to-bottom rationale for what the k3d
model should mirror, what we deliberately skip (supabase, kong,
oauth2-proxy, Studio — all unnecessary for knoe-auth's surfaces),
and the phasing. Captures the three scope decisions taken
2026-05-02 with the user:
- knoe-auth runs on the laptop, not as a pod (faster inner loop)
- stack scope is CNPG + KDC only
- single-replica CNPG, realm KNOE.LOCAL (distinct from KNOE.DEV)
Out-of-scope for Phase 1 listed explicitly so reviewer
expectations match scope: SPNEGO E2E, image-build-and-load,
supabase, oauth2-proxy, OidcCodeService DB persistence.
docs/plans/junie/k3d-knoe-auth-dev-loop.md
Phase 1 brief. Concrete deliverables:
- k8s/knoe/knoe-kdc-{deployment,service,configmap,pvc,init-job}.yaml
- etc/init_knoe_auth.sh --mode k3d flag (or sibling script)
- Makefile: k3d-knoe-{up,pf,down}
- etc/krb5.local.conf (with udp_preference_limit=1 for
kubectl port-forward UDP flakiness on macOS)
- docs/local-dev-knoe-auth.md (engineer-facing)
- scripts/k3d-knoe-smoke.sh
Definition of done: a fresh-clone laptop can `make k3d-knoe-up`
+ `make k3d-knoe-pf` + `mvn -pl authority spring-boot:run` and
hit /health, /.well-known/openid-configuration in <8 minutes.
Index updates:
docs/plans/README.md — k3d-gke-mirror.md row added
docs/plans/junie/README.md — split into Active / Shipped;
k3d brief listed under Active;
the 5 shipped-2026-05-02 briefs
moved to Shipped with commit refs.
docs/TODO.md In-progress — k3d brief now top of "In progress";
the previously in-flight "Phase 2
pg_oauth in install/deploy.sh"
item demoted to "paused" (resume
after the local dev loop lands).
No code changes; all docs.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Mirroring the 2026-04-29 DB-cluster removal. Live state showed garage
in an unconfigured limbo (layout v0, no role assigned, 21d of "Ring
not yet ready" warnings); nothing outside garage's own state
referenced its S3 endpoint anymore.
Removed from knoe-system namespace on knoe-dev-0:
- statefulset/garage (1/1 pod, 21d uptime)
- service/garage (ClusterIP; no garage-s3-ilb on app cluster)
- configmap/garage-config
- secret/garage-secrets
- pvc/data-garage-0 (29 Gi, garage-hdd)
- pv/pvc-656e1936-... (was Bound, reclaim=Retain)
- storageclass/garage-hdd
Underlying GCE pd-standard disk (us-west3-c/pvc-656e1936-...) reclaimed
via `gcloud compute disks delete` since reclaim=Retain leaves the disk
behind after the PV is gone.
Source manifests intact:
- k8s/knoe/garage-*.yaml (k3d, min)
- deploy/opentofu/k3s/manifests/knoe/garage-*.yaml (k3s)
- etc/init_garage_store.sh + knoe/core/ops/k8s_garage_store.py
- k8s/knoe/garage-statefulset-gcp.yaml (GKE variant)
A future ./install.sh against GKE could re-deploy garage if that's
what the deploy mode wires; whether it SHOULD wire garage given GCS
now handles backups is a separate cleanup decision.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Queue #5 Phase B orphan cleanup landed:
App cluster (knoe-dev-0):
- gitlab/gitlab-migrations-58a3e27-97-23-n72j8 (Failed/Evicted,
11d stale)
- pv/pvc-42e8545b-… (10Gi, monitoring/storage-kps-grafana-0)
- pv/pvc-86422179-… (30Gi, monitoring/prometheus-…-prometheus-0)
- pv/pvc-e2c29f7a-… (5Gi, monitoring/alertmanager-…-alertmanager-0)
DB cluster (knoe-dev-cnpg-0):
- kube-system/prole-supabase-dirprep Job (FailureTarget, 22d stale)
Post-state: 0 Released PVs on app cluster; gitlab namespace has only
the Completed migration pod; DB-cluster kube-system has no prole-*
artifacts. Live-cluster ops only; no source changes.
Queue #4 reframed: tried the planned `garage repair --yes blocks`,
but found garage in an unconfigured state — layout v0, no role
assigned, "Ring not yet ready" warnings continuous since whenever the
layout was lost (likely during a reconcile after the 2026-04-29 PVC
resize). Cross-cluster sweep shows nothing outside garage references
its S3 endpoint anymore (CNPG backups moved to GCS, no other workload
points at 10.180.15.239:3900). Garage on app-cluster is now a dead
pod holding an 80 GiB PVC. The original block-repair task is moot.
Reframed as a decommission-or-restore decision; details captured in
the ranked queue entry. Awaiting direction.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds a 2026-05-02 Done entry covering the cluster-side work that paired
with c3fae73 (source) and 03bb731 (chart cleanups): PodMonitor migration
applied with verified zero-data-loss handoff; supabase chart upgraded
revision 1→3; helm field-manager conflict on Kong configmap resolved;
node label drift on knoe-dev-0 closed (3 nodes labelled
knoe.dev/node-role=general alongside legacy prole.org/node-role); DB
cluster nodes still pending same treatment.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three Junie briefs landed in one commit:
#07 — Wire cnpg-backup-sa into CNPG cluster spec (drift R8)
deploy/gcp/gke/knoe-db.yaml: add spec.serviceAccountName: cnpg-backup-sa
(requires CNPG v1.29+, which is the live operator version).
etc/init_cnpg_gke.sh: operator install URL now uses CNPG_OPERATOR_VERSION
variable (default 1.29.0); new §11 patches knoe-db and
knoe-db-barman-cloud RoleBindings to add cnpg-backup-sa as a subject
if not already present — matching the 2026-04-29 live stabilization.
#13 — Migrate off deprecated enablePodMonitor + podMonitorRelabelings
Both deprecated fields removed from deploy/gcp/gke/knoe-db.yaml
spec.monitoring. New deploy/gcp/gke/knoe-db-podmonitor.yaml carries
the PodMonitor with the cluster relabeling rule (cnpg.io/cluster pod
label → cluster label; required for all 85 CNPG Grafana panels).
Apply alongside knoe-db.yaml on next cluster patch.
#15 — Remove dead DASHBOARD consumer + basicauth_credentials
supabase/helm/knoe-supabase:
- wrapper.sh: drop DASHBOARD_USERNAME / DASHBOARD_PASSWORD envsubst lines
- config.yaml: drop DASHBOARD consumer + basicauth_credentials block
- kong/deployment.yaml: drop both DASHBOARD env-var secret refs
- values.yaml: rename secret.dashboard → secret.openai (apiKey only;
username/password dropped — no enforcer since commit 25f1b2e)
- secrets/dashboard.yaml + _helpers.tpl: renamed to openai /
supabase.secret.openai
- studio/deployment.yaml: reads from secret.openai.apiKey
- ci/example.yaml: updated to secret.openai.apiKey
helm template confirms knoe-supabase-openai secret referenced; no
DASHBOARD output.
docs/TODO.md: queue items #7, #13, #15 + drift row R8 archived to Done.
Co-authored-by: Junie <junie@jetbrains.com>
Defects A, B, C from docs/plans/junie/06-patch-garage-script-fixes.md:
A. DB_CLUSTER default was knoe-cnpg-0 (stale rebrand artifact);
corrected to knoe-dev-cnpg-0 to match conf/gke.cfg and the
cluster table in CLAUDE.md. Header comment + overrides block
also updated.
B. Phase 1 deletion loop missed service/garage-s3-ilb (the
LoadBalancer the ObjectStore endpoint historically pointed at).
Now included alongside service/garage.
C. Phase 2 was applying knoe-db-backup-gcs.yaml whole, including a
legacy ScheduledBackup using method:barmanObjectStore (being
removed in CNPG v1.30). Manifest split handled in prior commit
(34a25dd); script's Phase 2 is now safe to re-run.
bash -n clean; CONFIRM=false dry-run prints knoe-dev-cnpg-0 correctly.
docs/TODO.md: queue items #2 and #6 + drift rows R5 and R9 archived to Done.
Closes queue item #6 in docs/TODO.md (drift R9).
Co-authored-by: Junie <junie@jetbrains.com>
Bringing the long-running session-feature branch back into main in one
deliberate sweep. The branch carried the cluster work that's been live for
weeks (cross-cluster CNPG metrics, Grafana w/ Google OAuth, supabase
oauth2-proxy, cluster recovery, pg.0.knoe.dev + per-engineer onboarding,
GCS-backed CNPG backups via Workload Identity, the env-contamination
guard, the Junie brief queue, the cnpg-grafana CSRF + memory-request
fixes from today), while main accumulated Junie's parallel knoe-auth
Phase 2 OIDC work (full provider surface: discovery, authorize, token,
userinfo, JWKS, RS256 signing, code exchange, session services).
Key decision: the two branches did COMPETING rebrands off the same
starting point (5ba9b63, 2026-04-27):
- claude branch (commit b355855, earlier): org.prole.authority.* →
dev.knoe.auth.*
(artifact renamed to
knoe-auth.jar)
- main (commit 9daa94b, recent): org.prole.authority.* →
dev.knoe.authority.*
(kept "authority" artifact name)
dev.knoe.auth wins: cluster runs from this name, the Maven artifact is
already knoe-auth.jar, and the broader rename is the documented
namespace direction (per ~/.claude/projects/-Users-chrisfu-dev-knoe-db/
memory/MEMORY.md). All of main's recent Phase 2 OIDC content was ported
from authority/src/.../dev/knoe/authority/ into
authority/src/.../dev/knoe/auth/ with package declarations rewritten.
== File-level resolution summary ==
Textual conflicts (4):
authority/pom.xml
- Took our artifactId="auth"
- Took our branch's removal of spring-security-kerberos-client
(verified: Junie's Phase 2 OIDC code does not import it; the dep
was already-dead config)
docs/pipeline-phases.md
- Took our branch's "Phase 1 not started" status. Main had a
misplaced "✅ Complete" with a knoe-auth-Phase-1 commit ref
in the autobuild Phase 1 section — different domain.
docs/plans/knoe-auth-round-1.md
- Took our branch's dev.knoe.auth file table (vs main's
dev.knoe.authority listing). Pure rename mismatch.
supabase/helm/knoe-supabase/templates/kong/config.yaml
- Took our branch's onboard route + plain dashboard wiring.
Main had an oauth2proxy.enabled toggle that put oauth2-proxy as
a Kong upstream — but the deployed architecture (commit 25f1b2e)
has oauth2-proxy in FRONT of Kong, not behind. Main's wrapper
reflected an architecture that was never deployed.
- Took our branch's removal of basic-auth from dashboard route
(queue #15 brief still tracks the matching values.yaml /
kong/deployment.yaml cleanup).
Java tree reconciliation (44 file-pairs):
20 dual-path source files + 2 dual-path tests
Body-identical between main's authority/ and our branch's auth/
after stripping package decls — main's commit 9daa94b was a pure
rebrand. Took our branch's auth/ version for all 22.
8 main-only source files (Phase 2 OIDC), ported into auth/:
web/JwksController.java
web/OidcAuthorizeController.java
web/OidcDiscoveryController.java
web/OidcTokenController.java
web/OidcUserInfoController.java
session/OidcCodeService.java
session/OidcTokenService.java
session/SessionService.java
12 main-only test files, ported into auth/:
HealthControllerTest.java
enroll/EnrollValueTypesTest.java
enroll/EnrollmentControllerTest.java
enroll/TotpServiceTest.java
kerberos/KadminClientTest.java
kerberos/KerberosSpnegoResultTest.java
web/LoginControllerTest.java
admin/AdminControllerTest.java
user/PrincipalNormalizerTest.java
regression/IdentityRegressionTest.java
session/OidcCodeServiceTest.java
session/SessionServiceTest.java
Port mechanics: read main:authority/...<file> via git show, then sed
rewrite `package dev.knoe.authority` → `package dev.knoe.auth` and
`import dev.knoe.authority` → `import dev.knoe.auth`. Body content
unchanged.
authority/src/main/java/dev/knoe/authority/ — DELETED (duplicate)
authority/src/test/java/dev/knoe/authority/ — DELETED (duplicate)
== Verification ==
- grep -rln '<<<<<<<' across .java/.md/.yaml/.yml/.sh/.xml/.tpl: clean
- find authority/src -path '*/dev/knoe/authority*': empty (subtree gone)
- grep 'package dev.knoe.authority' across repo: clean
- bash -n install.sh deploy.sh etc/preflight_kubecontext.sh: clean
- git ls-files -u | wc -l: 0 unmerged paths
- helm lint supabase/helm/knoe-supabase: pre-existing failure on
studioIngress.enabled undefined in values.yaml (introduced by Junie
on main; unrelated to this merge — flagging as follow-up).
== Followups (carried into TODO ranked queue or noted here) ==
- helm lint failure: studioIngress block in values.yaml is missing
enable flag; templates/studio/{ingress,oauth2proxy-deployment,
oauth2proxy-service}.yaml all reference studioIngress.enabled with
no default. Pre-existing on main; not introduced by this merge.
- The five Junie briefs filed on this branch are now reachable from
main at docs/plans/junie/{02,06,07,13,15}-*.md. Junie can pick them
up in any order.
- knoe-auth Phase 2 OIDC source (now at dev.knoe.auth.*) is not yet
deployed to the cluster. Deployment is its own task.
- The branch claude/crazy-bose-fec256 stays in place (worktree at
.claude/worktrees/crazy-bose-fec256 may have ongoing context for
Claude Code sessions). Safe to delete once next session starts
cleanly from main.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Filed in response to the 2026-04-28 14:00 UTC backup outage. An
`install.sh --mode k3d` run with the shell pointed at GKE silently
overwrote the GKE cluster's GCS-backed ObjectStore + ScheduledBackup
with k3d-mode defaults; Garage filled up and CNPG backups failed for
hours before the next manual check. The class of bug is "config says
target cluster A, shell context says target cluster B, installer
proceeds against B without warning."
New shared bash helper at etc/preflight_kubecontext.sh with two
functions:
- verify_kubecontext_matches_config <cfg-path>
Strict gate. Reads [Global] APP_CLUSTER_KUBECONTEXT from the
config and exits 1 if `kubectl config current-context` differs.
Skipped silently when the config has no baked APP_CLUSTER_KUBECONTEXT
(e.g. fresh k3d.cfg) or when there's no live current-context.
- print_kubecontext_notice
Informational. Prints what's about to be inherited so the user
can abort before the TUI launches if it looks wrong. Never fails.
Wiring:
- deploy.sh sources the helper and calls the strict gate against
${PROLE_DEPLOY_CFG:-conf/gke.cfg} before invoking Python.
Unattended path -> hard refusal on mismatch.
- install.sh sources the helper and calls the informational notice
(gated on not-`--min`) right after entering the local-checkout
branch. The TUI is interactive, so the strict mode-aware gate is
a follow-up once the welcome screen records a mode in
state.inputs.
Bypass for deliberate cross-cluster maintenance:
KNOE_SKIP_KUBECONTEXT_GUARD=true ./deploy.sh
End-to-end verified:
- deploy.sh with current=cnpg-0, gke.cfg=app-0 -> exit 1, clear msg
- deploy.sh with KNOE_SKIP_...=true -> bypasses, prints
"skipping check"
- install.sh --min -> notice skipped
- install.sh (no flag) and install.sh --silent -> notice printed
Doc updates:
- CLAUDE.md §"Env-contamination warning" rewritten to describe the
live guard (was a forward-looking TODO).
- CLAUDE.md drift table row R4 removed; "Closed 2026-05-01" line added.
- docs/TODO.md queue item #1 archived to Done; R4 dropped from the
reality-vs-intent table. Queue numbering retained (no #1 placeholder)
so the docs/plans/junie/<NN>-...md filenames still match.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Five self-contained work briefs in a new docs/plans/junie/ subdirectory,
each tagged against a numbered item in docs/TODO.md so Junie can take
them independently in any order.
02-k3s-prole-rename.md -> queue #2 (drift R5)
06-patch-garage-script-fixes.md -> queue #6 (drift R9)
07-init-cnpg-gke-sa-wiring.md -> queue #7 (drift R8)
13-podmonitor-manual-management.md-> queue #13
15-remove-dead-dashboard-consumer.md-> queue #15
Each brief follows the same shape: Why -> What changes (concrete file
paths + line numbers + before/after) -> Verification -> Out of scope ->
Commit shape -> Definition of done. The intent is that Junie reads cold
(no shared chat history) and lands the change without escalating
questions.
Also adds:
- docs/plans/junie/README.md describing the convention.
- Row in docs/plans/README.md so newcomers find the subdirectory.
- Brief reference + "Assigned to Junie" tag on each of the five queue
items in docs/TODO.md.
Existing in-progress assignment to Junie (Phase 2 pg_oauth) is unchanged.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Implement Google OIDC support in Authority module via GoogleOAuthService
- Update AuthProperties and application.yml with OIDC configuration
- Add oidc-setup.md documentation for GKE/Google Cloud setup
- Update etc/init_knoe_auth.sh to handle OIDC secrets and path-B configuration
- Configure knoe-auth-deployment.yaml and gke.cfg for production auth
Co-authored-by: Junie <junie@jetbrains.com>
Adds docs/plans/ as the canonical engineering reference for completed and
in-flight initiatives. Written for jr/mid engineers who have not seen the
repo before — each plan starts with strategic context and links to existing
code before asking for changes.
README.md index, audience, and status conventions
deployment-modes.md four-mode installer (min/k3d/k3s/gke), welcome-screen
mode selector, min-mode fast-path. Status: shipped.
knoe-auth-round-1.md Kerberos KNOE.DEV realm, invite-OTP enrollment,
Google corroboration, TOTP 2FA. Status: operational.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Delete tracked knoe-db/.prole_user_password (dead secret, rotated
in ansible since the original commit)
- Remove postgres-password.txt and build/ copy from working tree
- Add explicit ignore rules for *_password, postgres-password*,
.prole_user_password, *-secret variants
Add docs/plans/customer-deploy-resync.md with the upstream/downstream
migration plan and update CLAUDE.md to reflect prole's role as a
customer deploy of knoe-db.
Follow-up to the ansible excision: remove other artifacts that should
never have been tracked, and extend .gitignore so they do not come back.
Removed:
- ssl/prole/*.key, *.crt, *.srl -- prole.org-era private keys, registry
and DB CA material. References in knoe/core/env.py (auto-detect path
at :1488) are best-effort inside try/except, so deletion is safe.
- ssl/prole.ssl.tar -- tarball of the same material.
- deploy/opentofu/k3s/terraform.tfstate -- 114KB committed OpenTofu
state snapshot with live cluster UIDs. tfstate is never supposed to
live in git.
- docs/deploy_gke.log, docs/deploy_gke_done.log,
docs/compressed-history.log -- deploy-time log captures.
- end_time.txt, start_time.txt, start_time_final.txt -- ansible wall-
clock timing breadcrumbs.
- network_description.txt, network_prompt.txt -- local scratch prompts.
- sidekiq_logs.txt -- transient log capture.
- test_resolve.sh -- ad-hoc debug script.
.gitignore additions:
- *.tfstate, *.tfstate.backup
- /ssl/
- .ai/
- /end_time.txt, /start_time*.txt, /sidekiq_logs.txt,
/network_description.txt, /network_prompt.txt, /test_resolve.sh,
/docs/deploy_*.log, /docs/compressed-history.log
Note: the purged SSL keys remain in git history until the deferred
history-scrub pass (task #14). The repo will be created as Private in
git.knoe.dev so this is acceptable for now; it MUST be scrubbed before
any broader visibility change.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Documented step-by-step procedure for integrating GitLab with Google Workspace via OIDC (Path A).
- Outlined secret management patterns, GCP OAuth client setup, and deployment instructions.
- Included detailed remediation steps for secret leakage and Git history cleanup.
- Added verification and troubleshooting guides for pre- and post-deployment scenarios.
test: add helm monitoring tests for stale lock recovery and upgrade handling
- Introduced tests to validate Prometheus update flow with stale pending locks, including rollback and upgrade scenarios.
- Ensured robust handling for edge cases like missing deployed revisions and context mismatches.
Summarize CLAUDE.md updates:
- Document both live GKE clusters as e2-standard-2 x3 in us-west3 and note stale app default in code.
- Call out DB cluster naming mismatch between conf (knoe-dev-cnpg-0) and code default (knoe-cnpg-0).
- Add SSD quota guidance: 300GB fully consumed by CNPG, so non-CNPG PVCs must use standard/pd-standard.
- Refresh reset script reference and quota section details to match current runtime state.
Co-authored-by: Junie <junie@jetbrains.com>
Rename env config files from conf/*/prole.cfg to conf/k3d.cfg, conf/k3s.cfg, and conf/gke.cfg. Update shell/Python loaders and etc/deploy scripts to resolve named configs cleanly while keeping legacy fallback behavior. Align k3s Ansible tasks, docs, and regression coverage with the new configuration layout.
Co-authored-by: Junie <junie@jetbrains.com>
UI screens
- database.py: fix mode detection to use env_key priority (prod→k8s, service→k3s) so stale DEPLOYMENT_MODE never overrides the user's chosen environment
- database.py: Registry status reads ARTIFACT_REGISTRY_AVAILABLE persisted by cluster screen; uses SERVICE_NAMESPACE for Artifact Registry repo name
- cluster.py: add Artifact Registry traffic light (amber→green/red) to prod section; _check_artifact_registry_async persists ARTIFACT_REGISTRY_AVAILABLE into Global cfg
- cluster.py: re-trigger Artifact Registry check after GKE cluster selection so the light re-evaluates once region is available from KUBECONTEXT
- cluster_nodes.py: fix TclError on Python 3.14 — pady=(2,0) tuple → pady=2 scalar
- __init__.py: seed knoe-system namespace when saved value is "default", not only when empty
- services.py: replace hardcoded "Prole DB" log string with dynamic cnpg_cluster name
Core ops
- cloudnative_pg.py: replace one-shot Barman plugin retry with 6-attempt loop; first cert-manager/x509 failure triggers rollout restart + 30 s CA propagation wait; subsequent failures back off up to 60 s per attempt
- cloudnative_pg.py: TLS CA CN now uses cluster_name instead of hardcoded "Prole CNPG CA"
- registry.py, garage_store.py: refactored into per-mode modules (k3d/k3s/k8s registry and garage store, shared _garage_common)
Deploy / config
- deploy/gcp/gke/knoe-db.yaml: GKE-specific CNPG cluster manifest (rw/ro/r on separate nodes with premium-rwo storage)
- etc/init_common_services.sh, modes/k8s/knoe-db/.version: updated for current deploy
- kong-deployment.yaml: updated manifest
Tests
- test_cluster_nodes_render_smoke.py: add pack/grid, winfo_children, winfo_reqheight, update_idletasks, grid_slaves to dummy widgets; monkeypatch tk.Label so CNPG placement render completes without a real Tkinter root
Co-authored-by: Junie <junie@jetbrains.com>
- add tools/manage-node.sh for k3s node add/remove automation with inventory reconciliation
- include kubectl drain flag compatibility and --ssh-host targeting for uninstall workflows
- persist installer cfg/env values and update deploy/database/cfg UI handling
- refresh related inventory/network artifacts and expand installer/database test coverage
Co-authored-by: Junie <junie@jetbrains.com>
- Introduce mode-scoped directories under `~/.prole` for build contexts, runtime data, and versioning.
- Update DB image version generation and Dockerfile creation to respect deployment modes (e.g., k3d, k3s, k8s).
- Adjust UI, core actions, and scripts to avoid cross-mode conflicts and ensure correct version isolation.
- Add tests for mode-aware behaviors and update existing tests for new paths and markers.
- Bump PostgreSQL and image versions to `18` and `138`, respectively.
- Update Kubernetes manifests to align with new image and version configuration.
- Implement centralized `resolve_prole_home` utility for consistent environment-based `PROLE_HOME` resolution across modules
- Replace hardcoded home paths with `resolve_prole_home`
- Refactor PV management to support iSCSI mounts and node placement from Ansible manifests
- Improve Kubernetes manifest handling to dynamically apply namespaces per document
- Adjust `knoe-db` build context path and related tests
- Add utilities for detecting and applying Ansible-defined node labels and PVs
