Commit Graph

606 Commits

Author SHA1 Message Date
chrisfu
296bcd15a5 fix(dns,ssl): add gandalf A record; make prole_ssl resilient to missing certs
- Add gandalf.prole.org → 10.0.0.7 to prole_dns_records and
  prole_internal_a_records in dns.yml (was missing, causing nslookup
  to fail on myrddin)
- prole_ssl: stat-check each source file on the controller before
  copying; emit a warning and skip rather than hard-failing when certs
  have not yet been placed in ssl/prole/

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-05 20:22:22 -04:00
chrisfu
a575b608d6 Merge branch 'main' of git-ssh.knoe.dev:knoe.dev/knoe-db 2026-05-05 19:22:13 -04:00
chrisfu
e4478f20ab feat(tailscale): add Ansible role and playbook for merlin + gandalf
Installs Tailscale on k3s agents via the Debian stable apt repo using
the modern signed-by keyring approach. Auth key stored in vault.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-05 18:57:05 -04:00
chrisfu
9b2bf04067 fix(k3s): set GITEA_PV_NODE=gandalf.prole.org for k3s Gitea deploy
init_gitea.sh requires GITEA_PV_NODE in k3s mode. PVs for Gitea storage
(gitea-local-d005) are pinned to gandalf.prole.org per nodeAffinity on
the existing gitea-d005-shared-storage PV.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-04 06:00:11 -07:00
chrisfu
cb94788583 fix(kong): detect and delete orphaned svc-prole-ingress in legacy cleanup
cleanup_legacy_prole_kong() only checked for the prole-svc-kong deployment
and service, so the early-return guard fired after those were already deleted,
leaving svc-prole-ingress in place. On re-runs this Traefik ingress blocked
the duplicate-host check and prevented Kong from deploying.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-04 05:51:45 -07:00
chrisfu
4f3280002c fix(k3s): repair iscsi-pvs node names and kubectl context after prole→knoe rename
k8s/knoe/iscsi-pvs.yaml had *.knoe.org nodeAffinity after the rename, but the
live k3s cluster nodes are still *.prole.org — causing immutable field errors on
Bound PVs during kubectl apply. Revert to *.prole.org to match live state.

conf/k3s.cfg KUBECTL_CONTEXT was renamed to knoe-service-cluster, but both the
system kubeconfig and etc/secrets/k3s.kubeconfig still use prole-service-cluster.

install.sh _check_gcp_tools() now skips gcloud check for k3s mode and any non-gke
config path (only GKE deploys need gcloud/kubectl pre-flight).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-04 02:48:48 -07:00
chrisfu
2df7c91926 fix(kong): SERVICE_NAMESPACE=default in conf/dev overlay caused kong to deploy to default ns
conf/dev/knoe.cfg had a stale SERVICE_NAMESPACE=default written by a prior TUI session.
Also guard _service_namespace() so the value 'default' is never returned — it is never
a valid service namespace and signals a stale generated config.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-04 01:44:51 -07:00
chrisfu
c7871e2780 fix(installer): silent mode — gcp skip for k3d, op timeout, KNOE_SKIP_OP bypass
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-04 01:44:44 -07:00
chrisfu
a9b11f3ee6 fix(cnpg-backup/k3d): skip re-init when already healthy; add breadcrumb 2026-05-04 00:46:43 -07:00
chrisfu
de1a04c0fb fix(ui/cluster): stop combobox overlapping Add/Delete buttons
Combobox width=30 chars extends to ~320-380px depending on font metrics,
colliding with Add at x+290 and Delete at x+350. Narrowed combobox to
width=22 and shifted buttons to x+260 and x+320 to give clear separation.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-04 00:34:18 -07:00
chrisfu
2f98a5afd2 fix(cnpg/k3d): skip reinitialize when cluster already healthy
install_barman_plugin() fetches from GitHub on every run (up to 6×120 s
retries), stalling the installer worker thread even when the CNPG cluster
is already at 'Cluster in healthy state' with all pods 2/2 Ready.

Add a fast-path at the top of initialize(): check .status.phase for
'healthy' then verify all pods show 2/N ready — if both pass, return
immediately. Full pipeline (ensure_operator, install_barman_plugin,
_apply_manifest, _wait_cnpg_pods) is only entered when needed.

Also adds test_cnpg_initialize_skips_when_cluster_healthy to
test_deployment_mode_isolation.py to guard against regression.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-03 23:54:22 -07:00
chrisfu
5295e47753 fix(monitoring/k3d): skip helm upgrade when already deployed and healthy
kube-prometheus-stack CRD re-application stalls on every installer run
even when the release is in deployed state with all pods Running. For k3d
this is purely idempotency noise — skip the upgrade and return immediately.
Broken state (failed status, unbound PVCs) still triggers a full purge+reinstall.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-03 22:56:39 -07:00
chrisfu
47ae5d69d6 fix(supabase/k3d): disable node affinity, ingress, and TUI clutter
- render_supabase.py: k3d sets scheduling.enforceGeneralNodeRole=false so
  pods don't require knoe.dev/node-role=general label (k3d nodes unlabelled)
- render_supabase.py: disable kong and studio ingress for k3d (port-forwards
  are used instead; avoids db.knoe.org reference that doesn't exist
- deploy.sh: escape double quotes in SQL comment on lines 2114-2115 to fix
  bash prematurely closing the psql -c ... string
- supabase.py: remove broken node-selector text entry (black rectangle in Tk)
  and redundant Deploy button (enable checkbox already triggers deployment)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
EOF
)
2026-05-03 22:15:07 -07:00
chrisfu
59696889ea fix(supabase): use local-path storage class for k3d mode
k3d clusters only have local-path provisioner; defaulting to synology-iscsi
caused all Supabase PVCs to get stuck Pending. Also add local-path to the
validation allowlist so it is not rejected as an unknown storage class.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-03 21:51:06 -07:00
chrisfu
0008b0ad54 Merge branch 'main' of git-ssh.knoe.dev:knoe.dev/knoe-db 2026-05-03 21:13:49 -07:00
chrisfu
ae23dbb9f8 fix(knoe-users): use sh-compatible base64 pipeline (drop pipefail for dash)
The container's /bin/sh is dash, which doesn't support set -o pipefail.
The preceding [ -s /tmp/pg.keytab ] check already guarantees the file is
non-empty before this line runs, so pipefail was redundant.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-03 21:10:23 -07:00
chrisfu
d42dd583b5 chore(prole): homelab production config — prole.org fork
conf/service/prod.cfg: prole.org homelab overrides
  - SERVICE_HOST_DOMAIN=prole.org, DB_UI_HOST=db.prole.org
  - OIDC_ISSUER_URL=https://api.prole.org/auth
  - GOOGLE_WORKSPACE_DOMAIN=prole.org (credentials in OpenBao bao/kv/knoe/oauth2/google-prole-org)
  - kerberos_config: enabled, realm=PROLE.ORG, kdc=10.0.0.3 (myrddin.prole.org)
  - PROLE_KDC_TRUST_REALM=PROLE.ORG, SERVICE_INGRESS_TLS_ENABLED=1

conf/k3d.cfg: update local dev paths to ~/dev/prole; enable kerberos for dev

env.sh: KNOE_HOME points to ~/dev/prole

application-prole.yml: Spring profile with prole.org Google Workspace client binding

These files are prole.org homelab specifics — do NOT merge upstream to knoe/main.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-03 20:03:20 -07:00
chrisfu
bb71cf68ea feat(auth): dual IdP for db.prole.org — Google OAuth + Kerberos SPNEGO
knoe-auth (Spring Boot OIDC provider):
- AuthProperties: add google.workspaceDomain and kerberos.servicePrincipal fields
- GoogleOAuthService: validate hd (hosted domain) claim; restrict to configured workspace
- LoginController: /login/google endpoint + SPNEGO negotiation entry point
- PrincipalNormalizer: map Kerberos principal (user@REALM) to knoe-auth user
- application.yml: add spring.security.kerberos and oauth2.client stubs (values
  injected at runtime from env / Kubernetes Secrets)
- knoe-auth-deployment.yaml: mount HTTP keytab Secret; add GOOGLE_PROLE_CLIENT_ID /
  GOOGLE_PROLE_CLIENT_SECRET env from oauth2-proxy-prole-secret
- knoe-auth-http-keytab-secret.example.yaml: example Secret for HTTP/<host> keytab

Kong (init_kong.sh):
- Add db.prole.org route in k3s mode block via oauth2-proxy upstream
- Mode-gate: only registered for k3s, excluded for k3d/k8s

Supabase / oauth2-proxy:
- New supabase/helm/oauth2-proxy Helm chart: gates Supabase Studio at db.prole.org
  with Google OAuth (email-domain=prole.org) + cookie settings for .prole.org domain
- values-k3s.yaml: k3s-specific overrides (upstream service, TLS, cookie domain)
- secret-example.yaml: placeholder for oauth2-proxy-prole-secret

Ansible:
- infrastructure/playbooks/kerberos_trust_setup.yml: automates samba-tool domain
  trust create on myrddin.prole.org for PROLE.LOCAL ↔ PROLE.ORG cross-realm trust

Test:
- GoogleLoginProleOrgTest: verifies hd=prole.org tokens are accepted; hd=other.com rejected

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-03 20:03:09 -07:00
chrisfu
30c7bc88fd fix(knoe-users): keytab reliability + mode-aware next steps + Gitea auto-token
Keytab export (silent failure bug):
- Always delete+recreate the postgres SPN with fresh random key; old key gone
  after EmptyDir wipe; kadmin.local -q exits 0 even on error so -norandkey
  silently failed
- Clean /tmp/pg.keytab before ktadd; verify non-empty with [ -s ] before
  proceeding; die loudly if keytab not written
- Fix base64 pipeline: set -o pipefail inside sh -c so base64 failure is not
  masked by tr exit code

Next steps:
- Suppress myrddin.prole.org Samba trust step in k3d mode (no AD server)
- Remove Grafana auth.proxy reminder (configured by Helm values already)

Gitea admin token:
- Auto-generate via kubectl exec into running Gitea pod before falling back to
  manual warning; persist as gitea-admin-token secret for future re-runs

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-03 16:34:03 -07:00
chrisfu
9d7ef668b1 fix(knoe-users): fix KDC re-init pod selection and add readiness poll
Two bugs in the EmptyDir auto-recovery path:

1. get_kdc_pod returned items[0] without a phase filter, so a Terminating
   pod from the previous rollout could be selected. Add
   --field-selector=status.phase=Running to always get a live pod.

2. The rollout completes (pod Running) before the in-container entrypoint
   finishes kdb5_util create (no readiness probe configured). The single
   immediate kadmin.local check raced against DB init and lost. Replace
   with a 120s poll loop (5s interval) that re-queries the pod each tick
   so it handles both the timing race and any remaining pod-selection lag.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-03 16:10:39 -07:00
chrisfu
e25090e6e1 fix(knoe-users): re-query KDC pod after re-init before retrying pre-flight
init_kdc.sh initialize rolls out a new deployment, creating a new pod with a
new name. The retry pre-flight was still exec-ing into the old (terminated)
pod. Re-query get_kdc_pod() after re-init so the retry and all subsequent
kadmin calls use the new pod.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-03 15:46:20 -07:00
chrisfu
7466bbee3a fix(knoe-users): auto-recover KDC database when pod restarted with EmptyDir
When the KDC pod restarts (e.g. after k3d node restart to apply registry
mirror config), its EmptyDir volume is wiped. ensure_kdc_pod() finds the
running pod and returns it, bypassing the bootstrap path. The subsequent
pre-flight check then fails because kadmin.local cannot access the database.

Instead of dying with a manual-intervention message, auto-recover by calling
init_kdc.sh initialize (with PROLE_KDC_NAMESPACE/NAME/MASTER_PASSWORD set)
then retrying the pre-flight. Only die if the retry also fails.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-03 15:38:18 -07:00
chrisfu
96964b0a2b fix(check_kerberos): default PROLE_KDC_NAME to authority-knoe-auth (matches init_kdc.sh)
check_kerberos.sh defaulted PROLE_KDC_NAME=auth but init_kdc.sh deploys the
pod with label app=authority-knoe-auth (authority-gcp-auth for k8s). The pod
selector never matched, so wait_for_kdc_pod always returned empty and the test
failed with "KDC pod 'auth' not found" even after a successful rollout.

Add _default_kdc_name() using the same mode-aware logic as init_kdc.sh.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-03 15:23:40 -07:00
chrisfu
d110f594c6 fix(k3d): patch containerd registry mirror on all cluster nodes
k3d registry containers are addressed as k3d-NAME.localhost:PORT from
inside the cluster, but containerd resolves that as 127.0.0.1 (loopback)
which has nothing listening.  The fix is to write /etc/rancher/k3s/registries.yaml
on every server/agent node to mirror k3d-NAME.localhost:PORT -> http://k3d-NAME:PORT
(Docker network DNS), then restart the node so containerd picks it up.

_patch_k3d_registry_mirrors() is called from _ensure_k3d_registry() after
the registry is confirmed running (idempotent — skips nodes already configured).
This makes knoe-authority and any other image pushed to localhost:5000
immediately pullable inside the cluster as k3d-knoe-registry.localhost:5000/...

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-03 15:18:11 -07:00
chrisfu
e75e03fb3f fix(rebrand): rename prole_*.sh shell libs to knoe_*.sh
check_kerberos.sh (already updated) sourced knoe_env.sh, knoe_secrets.sh,
knoe_string.sh, and knoe_yaml.sh but the files were still named prole_*.sh.
No other script referenced the old names, so this is a pure rename.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-03 14:57:09 -07:00
chrisfu
63185d8104 fix(port-forwards): make argocd/dashboard conditional; fix grafana release name; non-fatal start
Port-forward setup was failing the installation because:
1. argocd and dashboard entries were always added to port-mapping even when
   those services haven't been selected or installed yet
2. Grafana/Prometheus service names were hardcoded as kps-* but monitoring
   deploys with release name "prometheus" → services are prometheus-grafana etc.
3. The grafana status poll after init_port_forwards.sh start would mark the
   entire installation failed if grafana wasn't reachable

Changes:
- _build_required_port_forwards (env.py): add argocd_enabled, dashboard_enabled,
  monitoring_release params; argocd/dashboard only included when enabled; grafana
  and prometheus targets use f"svc/{monitoring_release}-grafana" etc.
- _sync_port_forward_mappings (actions.py): read argocd_enabled from installer
  config; read MONITORING_RELEASE from Monitoring section; clean up argocd and
  dashboard entries from [Port Forwards] when disabled
- run_init_scripts (services.py): port-forward step is non-fatal — failure logs
  a warning and continues; blocking grafana poll removed
- 2 new regression tests (7e) covering the above invariants

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-03 14:39:10 -07:00
chrisfu
c75c63eb9c fix(services): pass service namespace to init_kong.sh, not DB namespace
services.py run_init_scripts() called init_kong.sh with -n env[NAMESPACE]
where NAMESPACE is the database namespace (e.g. knoe-db). The -n flag
sets COMMON_CORE_NAMESPACE which wins over SERVICE_NAMESPACE in
common_core_resolve_namespace, so Kong deployed into knoe-db.

Fix: resolve kong_ns from SERVICE_NAMESPACE, falling back to
_get_service_namespace() and finally knoe-system. Exclude default
as it is the Tk StringVar initial value, not a real service namespace.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-03 13:56:34 -07:00
chrisfu
4bcd8f846e fix(milestone): SERVICE_NAMESPACE must not fall back to DB namespace
When SERVICE_NAMESPACE was absent from the config file and OS env,
_get_script_env() fell back to env["NAMESPACE"] — the database namespace
(e.g. knoe-db). InitializationScriptsMilestone then called init_kong.sh
with SERVICE_NAMESPACE=knoe-db, causing Kong to deploy into the DB
namespace and collide with the existing knoe-system/svc-knoe-ingress.

Fix: default to "knoe-system", matching _service_namespace() in actions.py.
Configs that set SERVICE_NAMESPACE explicitly are unaffected.

Add tests/test_deployment_mode_isolation.py to guard against k3d/k3s/gke
mode-specific config bleeding into each other: SERVICE_NAMESPACE fallback,
monitoring storage class separation, Kong gitea host gating, and
_service_namespace() across all three deployment modes.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-03 13:22:12 -07:00
chrisfu
8dcec846cd fix(monitoring): stop purge cycle on k3d retry
Three issues caused the purge-and-reinstall loop:

1. _purge_broken_monitoring deleted ALL PVCs including Bound ones; grafana's
   working PVC was wiped on every retry. Now only delete PVCs in Pending state.

2. Broken-state detection keyed on Pending pods + unbound PVCs, which is true
   during any still-converging install (including ones that timed out but
   whose pods eventually came up). Gate on helm status=='failed' + unbound PVCs.

3. k3d install used --wait, which blocks on all kube-prometheus-stack components
   (prometheus, alertmanager, node-exporter). They converge async after the
   Prometheus operator starts; --wait always timed out. Drop --wait for k3d;
   the status_common_services.sh check verifies readiness independently.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-03 12:55:11 -07:00
chrisfu
489593dca2 fix(monitoring): extend helm --wait timeout to 20m for k3d
kube-prometheus-stack pulls multiple large images on a fresh k3d install;
600s / 10m is not enough. Set --timeout 20m and process timeout 1320s
(20m + 2m buffer) for k3d, keep 10m for k8s/k3s where images are warm.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-03 01:23:06 -07:00
chrisfu
ac7b5928ca fix(monitoring): k3d values + purge broken release with stuck PVCs
Two bugs kept prometheus-grafana-0 Pending for 21 days in k3d:

1. update() routed k3d through _values_yaml_k3s() which hardcodes
   merlin-local-iscsi-* storage classes and merlin.prole.org node affinity
   — neither exists in k3d, so grafana's PVC could never bind.
   Add _values_yaml_k3d() that uses local-path (or MONITORING_STORAGE_CLASS
   override) with no node affinity; dispatch k3d separately in update().

2. No broken-state recovery for non-k8s mode. Add _purge_broken_monitoring()
   (helm uninstall + delete stuck PVCs) and call it from update() when
   Pending pods and unbound PVCs are both detected in the monitoring namespace.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-03 01:03:59 -07:00
chrisfu
e10d4f3782 fix(k3d): fix prole-registry squatting port 5000 in init_registry.sh
The shell path (init_registry.sh apply_registry) is the real k3d registry
entry point. It checked 'k3d registry list knoe-registry' which returned 0
because k3d-knoe-registry existed in 'created' state, so the create was
skipped and k3d-prole-registry (the port-5000 squatter) was never touched.

- init_registry.sh: before the knoe-registry ensure, detect k3d-prole-registry
  via both k3d and docker inspect, then stop/rm the Docker container regardless
  of whether k3d manages it; also detect knoe-registry in non-running state and
  nuke+recreate it (port was unavailable on the previous attempt)
- k3d_registry.py: same cleanup logic for the Python fallback path

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-03 00:43:54 -07:00
chrisfu
055069c1f8 fix(k3d): clean up prole-registry and fix registry running check
k3d-prole-registry was squatting on port 5000, leaving k3d-knoe-registry
in created state. Two bugs made this invisible:

- k3d_registry.py: substring match (name in stdout) matched k3d-knoe-registry
  as already-exists without checking STATUS=running; add _k3d_registry_running()
  requiring last column == running, detect and delete prole-registry before
  creating knoe-registry, recreate if found in non-running state
- status_common_services.sh: grep -qx 'knoe-registry' (exact) never matched
  k3d prefix 'k3d-knoe-registry'; fix with awk suffix match + STATUS==running

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-03 00:31:43 -07:00
chrisfu
82c1ff555f fix(k3d): rename prole-svc-kong → knoe-svc-kong, clean up stale resources
k3s manifests still used prole-svc-kong after the rebrand; init_kong.sh
was waiting on `rollout status deployment/knoe-svc-kong` which never
existed, hanging the Common Services milestone.

- Rename kong-deployment.yaml and kong-service.yaml: prole-svc-kong →
  knoe-svc-kong (labels, selector, configmap volume ref)
- Remove prole-svc-kong-configmap.yaml (static file replaced by
  init_kong.sh dynamic ConfigMap generation; hardcoded namespace and
  prole-branded routes were dead weight)
- init_kong.sh: add cleanup_legacy_prole_kong() called from action_update()
  to remove stale prole-era resources before deploying knoe-svc-kong
- init_kong.sh: add _is_host_claimed_by_other_ingress() helper; set
  include_gitea_host=0 for k3d mode (git is port-forward only, no public
  hostname) — keeps the safety-net pre-check for k3s
- actions.py: extend legacy namespace dedupe to sweep prole-svc-kong from
  both default and service_ns

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-03 00:24:02 -07:00
chrisfu
dba8a2d1dc feat(installer): TDD stabilization for k3d install path; dual-cluster GKE TUI
Junie's session targeted the prompt "stabilize ./install.py -c conf/k3d.cfg
using strict TDD" — broad installer-side work, not the k3d-mirror Phase 3
brief I had filed (which she didn't pick up; phase-3 brief stays open). All
750 installer tests pass post-change.

What Junie produced:

  install.py                          (NEW) Top-level CLI entry point. Was
                                            imagined by the prompt but didn't
                                            exist; this commit makes it real.
  knoe/deployment.py                  (NEW) `KnoeDeployment` orchestrator for
                                            the k3s service-mode deploy pipeline.
                                            Wraps Ansible kubeconfig fetch,
                                            opentofu apply, init_*.sh post-apply
                                            scripts, and (optionally) supabase/
                                            deploy.sh.
  knoe/ui/screens/cluster.py          Dual-cluster GKE kubecontext UI: prod env
  knoe/ui/screens/cfg.py              now shows separate "App Cluster:" and
                                       "DB Cluster:" dropdowns instead of a
                                       single "Kubernetes Context:" combo.
                                       New _app_kubectx_combo + _db_kubectx_combo
                                       widgets; new app/db_cluster_kubecontext
                                       tk.StringVars.
  knoe/core/{actions,env,milestones}.py
  knoe/core/ops/storage.py
  knoe/config.py, knoe/knoe_conf.py   Plumbing changes for the dual-cluster
                                       kubecontext flow + storage-class topology
                                       detection cleanup.
  knoe/tools/cleanup_cnpg_storage.py  (NEW) Stand-alone cleanup utility.
  tools/dashboard.sh                  (NEW) Dashboard helper.
  conf/knoe.cfg                       (NEW) Master cfg generated by knoe_conf.
  conf/dev/                           (NEW) Dev-mode cfg directory.
  conf/port-mapping.cfg               Port mapping tweaks for k3d.
  tests/installer/* (8 files)         New + extended tests for the dual-cluster
  tests/test_database_options.py      TUI, kubecontext save flow, storage ops,
                                       topology detection, deploy helpers,
                                       database-options screen.

Issues found in Junie's working state and fixed here:

  1. install.py was a 11-line import shim with no shebang, no `chmod +x`,
     no `if __name__ == '__main__'` block. `./install.py -c conf/k3d.cfg`
     returned `Permission denied` and `python install.py` did nothing.
     Added `#!/usr/bin/env python3`, `chmod +x`, and a __main__ block
     that delegates to `knoe.ui.screens.main()`. `./install.py --help`
     now prints the canonical argparse help.

  2. knoe/deployment.py had FIVE `subprocess.run()` call sites with no
     `timeout=` argument (`_run_script`, `_run_cmd`, the Ansible playbook
     fetch, `tofu init`, `tofu apply`). A hung child process — typical
     failure mode is a script waiting on stdin or a stalled network
     call — would lock up the installer indefinitely. Added timeouts:
       - Ansible kubeconfig fetch: 120s
       - tofu init: 300s
       - tofu apply, _run_script, _run_cmd: bounded by new module
         constant `_MILESTONE_TIMEOUT` (default 1800s = 30 min, override
         via `KNOE_MILESTONE_TIMEOUT_SECONDS` env var).
     `subprocess.TimeoutExpired` is caught explicitly; on timeout the
     run helpers return exit code 124 (conventional timeout code).

  3. `conf/k3d.cfg` was corrupted with MagicMock string-reprs on disk:
        KNOE_CONF = <MagicMock name='Canvas().tk.call().strip()' id='4743999712'>
        argocd.node_selector = <MagicMock name='mock.StringVar().get().strip()' id='...'>
     Likely path: Junie ran `./install.py -c conf/k3d.cfg` interactively
     in a non-Tk environment (or with a partially-mocked widget set) and
     the installer's "save current state" path wrote the mock-objects'
     `__repr__` strings into the cfg file. This commit reverts the cfg
     to its pre-Junie state. **Followup: harden the cfg save path
     against non-string widget values** — track separately.

  4. The corrupted cfg caused the installer to call `os.makedirs()` on
     the mock-string values, producing 10 directories on disk literally
     named `<MagicMock name='Canvas().tk.call().strip()' id='4733210304'>/`
     etc., with 5–86 files of install artifacts inside each. Removed.

The "final step is timing out" the user reported was almost certainly
issue #2 above: install.py walked the milestone pipeline, hit one of
the unbounded subprocess.run calls, and the wrapped command (probably
supabase/deploy.sh, which Junie was reading for context when her
session timed out) hung. With the timeouts in place that path now
exits cleanly with rc=124 instead of locking up.

Verification:
  - pytest tests/installer/ -q                                   750 passed in ~25s
  - python3 -c "import knoe.deployment"                          imports clean
  - ./install.py --help                                          prints argparse help
  - find . -maxdepth 1 -type d -name '<MagicMock*' | wc -l       0
  - head -7 conf/k3d.cfg                                          clean (no MagicMock)

Out of scope for this commit (followups):
  - The cfg save-path that wrote mock-objects-as-strings (issue #3 root cause).
    Reproducer: launch the installer in an env where Tk widget vars are
    `unittest.mock.MagicMock` instances. The cfg save code should refuse to
    serialize non-str values rather than calling `str()` on a MagicMock.
  - The k3d-mirror Phase 3 brief (`docs/plans/junie/k3d-knoe-auth-pod-deploy.md`)
    is still open — Junie picked a different prompt this round.

Co-authored-by: Junie <junie@jetbrains.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-02 23:51:14 -07:00
chrisfu
2b36add592 docs(plans): file Phase 3 brief — knoe-auth as a pod inside k3d
Phase 1 (host loop) and the Phase 2 OIDC sandbox are shipped; this
brief queues Phase 3 of k3d-mirror-of-GKE: build the knoe-auth image,
k3d-import, run as a pod inside the cluster. Pre-merge smoke loop —
not the daily inner loop.

Deliverables (see brief for the full Definition of done):
  - k8s/knoe/knoe-auth-deployment.yaml         (NEW; sibling of GKE manifest)
  - scripts/k3d-knoe-deploy.sh                 (NEW; build + import + apply + wait)
  - Makefile k3d-knoe-{deploy,redeploy,undeploy} targets
  - Phase 2 signing key flows from etc/secrets/knoe-auth-oidc-key.b64 into
    a knoe-auth-oidc-signing-key K8s Secret in knoe-system, mirroring how
    the GKE deploy reads it.

Index updates:
  docs/plans/junie/README.md           — k3d Phase 3 in Active row
  docs/TODO.md §"In progress"          — promotes the brief to top
  docs/plans/k3d-gke-mirror.md         — banner + §6 Phase 3 entry
                                          flipped from "out of scope"
                                          to "in flight"
2026-05-02 13:15:29 -07:00
chrisfu
903f84f200 feat: ship Junie #3 (image rename) + Phase 2 OIDC GKE deploy + k3d chrisfu seed
Three independent lines of work landing in one commit because they were all
on disk together end-of-session and the cross-file edits (TODO, knoe-system,
junie/README) interleave cleanly:

1. **Queue #3 — image rename `knoe-authority` → `knoe-auth`** (Junie).
   Closes drift R6. The Maven artifact has been `knoe-auth.jar` since
   commit b355855; the deploy manifests now match.
     authority/Dockerfile.app                                         (NEW)
     deploy/gcp/gke/knoe-auth-deployment.yaml                         (3 image tags renamed)
     deploy/opentofu/k3s/manifests/knoe/knoe-auth-deployment.yaml     (2 image tags renamed)
     Makefile                                                         (docker-build-auth +
                                                                       docker-push-auth + REGISTRY/KNOE_AUTH_VERSION
                                                                       defaults)
     docs/plans/junie/03-image-rename-knoe-authority-to-knoe-auth.md  (brief, kept as design record)

2. **Phase 2 OIDC provider — GKE deploy** (Junie). Source landed via the
   merge that brought claude/crazy-bose-fec256 back; the k3d sandbox
   shipped earlier today (commit 93157b0). This commit completes the GKE
   path: Kong route `/auth` → knoe-auth, default values added, signing
   key wired via `knoe-auth-oidc` secret per the brief.
     supabase/helm/knoe-supabase/templates/kong/config.yaml           (Kong route /auth, gated
                                                                       on .Values.knoeAuth.enabled)
     supabase/helm/knoe-supabase/values.yaml                          (knoeAuth.enabled=false default;
                                                                       studioIngress block — also
                                                                       fixes the helm lint issue noted
                                                                       in earlier commit 03bb731)
     docs/plans/junie/phase2-oidc-gke-deploy.md                       (brief, kept as design record)

3. **k3d dev-user seed: `chrisfu` + `knoe_developer`** (Claude). Closes
   "I want auth as chrisfu@knoey.com to my local cnpg database" for the
   local dev loop. `etc/init_knoe_auth.sh --mode k3d` now runs
   `seed_dev_users_k3d()` after the schema bootstrap, creating:
     - `knoe_developer` group role with R/W on `knoe`+`public` (mirrors
       the GKE production layout from docs/db-access.md; was hand-rolled
       in production per the 2026-04-30 onboarding work, never baked into
       postInitTemplateSQL).
     - `chrisfu` LOGIN role with password `chrisfu-dev` (idempotent —
       resets on every `make k3d-knoe-up`, so the rebuild loop is
       deterministic). Granted into `knoe_developer`.
   From the host with port-forward up:
     PGPASSWORD=chrisfu-dev psql "postgresql://chrisfu@localhost:5432/knoe-db?sslmode=require"

Cross-cutting doc updates:
  docs/TODO.md            — Phase 2 GKE entry removed from "In progress"
                            (now "(none — all items shipped or paused)");
                            queue #3 + reality table R6 removed; pg_oauth
                            paused note flipped to "OIDC issuer now
                            deployed to GKE — pg_oauth can resume"; Done
                            section gets new entries for #3, Phase 2 GKE,
                            and chrisfu seed.
  docs/knoe-system.md     — Phase 2 GKE row → Shipped; pg_oauth → Ready;
                            knoe-auth-deployment.yaml note "knoe-auth:latest"
                            instead of "knoe-authority:latest"; "One-time
                            cluster setup" mentions the chrisfu seed.
  docs/local-dev-knoe-auth.md — "Verify psql connectivity" rewritten to
                            connect as chrisfu (was `knoe`); new "Reset /
                            rebuild loop" section explaining idempotency.
  docs/plans/junie/README.md — two new rows under Shipped: brief 03 and
                            phase2-oidc-gke-deploy.

Verification (the bits I ran locally):
  - bash -n etc/init_knoe_auth.sh                                     OK
  - The seed SQL is idempotent (DO blocks with EXISTS checks +
    ALTER ROLE on the password reset path)
  - The Kong /auth route is gated on .Values.knoeAuth.enabled — default
    false, no behavior change until someone flips it on per Junie's brief

Out of scope for this commit:
  - Actually flipping knoeAuth.enabled=true on the live GKE chart and
    rolling out — Junie's brief covers the runbook; needs the
    knoe-auth-oidc K8s secret populated from 1Password first
  - Rebuilding + pushing the new knoe-auth:latest image — `make
    docker-push-auth` is wired, just hasn't been run yet
  - Round 1.5 OpenBao transit-key encryption (still queued)

Co-authored-by: Junie <junie@jetbrains.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-02 13:07:40 -07:00
chrisfu
93157b0a86 feat(knoe-auth): Phase 2 OIDC sandbox in k3d dev loop
Completes the Phase 2 OIDC laptop dev path. Source for the OIDC surface
(discovery, authorize, token, userinfo, JWKS controllers + signing /
session services) landed on `main` via the merge that brought
claude/crazy-bose-fec256 back. This commit makes Phase 2 actually
exercisable in the k3d dev loop without any GKE deploy.

What's new:

  authority/src/main/resources/application-k3d.yml
    Spring profile activated by `-Dspring-boot.run.profiles=k3d`.
    Enables OIDC (knoe.oidc.enabled=true), points the issuer at
    `http://localhost:8080`, sets Kerberos realm to KNOE.LOCAL, and
    aligns the datasource with the port-forwarded localhost:5432 DB.

  etc/gen_oidc_signing_key.sh  (executable)
    Idempotent RS256 PKCS#8 keypair generator. Outputs:
      etc/secrets/knoe-auth-oidc-key.pem  (PEM)
      etc/secrets/knoe-auth-oidc-key.b64  (single-line base64 of DER —
        directly consumable as KNOE_AUTH_OIDC_SIGNING_KEY by
        OidcTokenService.init())
    `etc/secrets/` is already gitignored. Set FORCE=1 to rotate.

What's wired:

  scripts/k3d-knoe-up.sh
    New §7 calls etc/gen_oidc_signing_key.sh after schema seed. Keypair
    persists across `make k3d-knoe-down && make k3d-knoe-up` cycles.

  scripts/k3d-knoe-pf.sh
    Output now includes the KNOE_AUTH_OIDC_SIGNING_KEY export line, the
    full `mvn spring-boot:run` invocation with -Dspring-boot.run.profiles=k3d,
    and the three OIDC endpoints to curl-test.

What's documented:

  docs/local-dev-knoe-auth.md
    "Daily loop" Terminal B: now exports KNOE_AUTH_OIDC_SIGNING_KEY,
    runs with `-Dspring-boot.run.profiles=k3d`, and the verify section
    includes /jwks.json. IntelliJ run config: adds Active Profiles: k3d
    and a note about pasting the b64 directly (no shell expansion in
    the env-var field).

  docs/knoe-system.md
    Phase 2 status row split: "k3d setup" → Shipped, "GKE deploy" →
    Pending. The "Open work items" Phase 2 entry rewritten to flag
    that the GKE deploy is the remaining thread (gated on queue #3 for
    the image rebuild as `knoe-auth:latest`).

  docs/TODO.md
    Promoted "Phase 2 OIDC provider — GKE deploy" into §In progress
    (replacing the empty "(none)" placeholder). Done section updated
    with two entries: the k3d Phase 1 dev loop (Junie's c3...) and
    this Phase 2 OIDC k3d sandbox.

End-to-end loop the engineer can run:

  make k3d-knoe-up                                     # one-time, ~5 min
  make k3d-knoe-pf &                                   # port-forwards
  export KRB5_CONFIG=$PWD/etc/krb5.local.conf
  export KNOE_AUTH_OIDC_SIGNING_KEY=$(cat etc/secrets/knoe-auth-oidc-key.b64)
  mvn -pl authority spring-boot:run \
    -Dspring-boot.run.jvmArguments="-Djava.security.krb5.conf=$PWD/etc/krb5.local.conf" \
    -Dspring-boot.run.profiles=k3d
  # then:
  curl -s http://localhost:8080/.well-known/openid-configuration | jq .issuer
  # → "http://localhost:8080"
  curl -s http://localhost:8080/jwks.json | jq '.keys[0].kty'
  # → "RSA"

Verified locally: keypair generator round-trips through openssl pkey -inform DER
(produces valid 2048-bit RSA keys); idempotent (existing key kept by default,
FORCE=1 rotates); bash -n clean on all 5 touched scripts.

Out of scope (TODO §In progress captures it):
  - GKE deploy of Phase 2 (image rebuild + K8s Secret + deployment env vars)
  - SPNEGO E2E from host browsers (k3d-mirror Phase 2)
  - knoe-auth-as-pod in k3d (k3d-mirror Phase 3)
  - OidcCodeService DB persistence (separate track)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-02 11:55:50 -07:00
chrisfu
5d3600845d feat(k3d): laptop dev loop for knoe-auth — CNPG + KDC + port-forward
Brings up the smallest k3d-resident stack that lets a host-side knoe-auth
(run via `mvn spring-boot:run` or IntelliJ) iterate against real Postgres
+ Kerberos. Closes Phase 1 of the k3d-gke-mirror plan (docs/plans/k3d-gke-mirror.md).

Scope:
  - k8s/knoe/knoe-kdc-{configmap,deployment,service,pvc,init-job}.yaml
    NEW; standalone KDC, realm KNOE.LOCAL (distinct from KNOE.DEV).
  - etc/init_knoe_auth.sh: --mode k3d flag added; swaps realm + skips
    GCP-specific steps. GKE behavior unchanged when flag absent.
  - Makefile: k3d-knoe-up, k3d-knoe-pf, k3d-knoe-down (delegate to
    scripts/k3d-knoe-{up,pf,down}.sh).
  - scripts/k3d-knoe-{up,pf,down,smoke}.sh NEW; up = full bring-up,
    pf = three port-forwards (5432/88/464) + JDBC URL + ^C cleanup,
    down = teardown, smoke = sanity check.
  - etc/krb5.local.conf NEW; checked-in libdefaults+realms config
    pointing at localhost:88. udp_preference_limit=1 to dodge
    kubectl port-forward UDP flakiness on macOS.
  - docs/local-dev-knoe-auth.md NEW; one-time setup + daily loop +
    IntelliJ run config.
  - docs/knoe-system.md NEW; unified reference for the knoe-auth
    service (GKE deployment + k3d dev loop + schema overview +
    source map + open work items).

Verified per the brief's Definition of done: fresh-clone laptop can
`make k3d-knoe-up` + `make k3d-knoe-pf` + `mvn -pl authority spring-boot:run`
and hit /health, /.well-known/openid-configuration in <8 minutes.

Out of scope (parent plan docs/plans/k3d-gke-mirror.md §6):
  - SPNEGO from host browsers (Phase 2)
  - knoe-auth-as-pod / image build/load (Phase 3)
  - Supabase stack on k3d (Phase 4)
  - OidcCodeService DB persistence (separate track)

docs/plans/junie/README.md — k3d brief moved from Active to Shipped.
docs/TODO.md — In-progress now empty; Phase 2 pg_oauth notes that the
local dev loop is in place so it can resume.

Closes Phase 1; Phase 2+ briefs filed as needed.

Co-authored-by: Junie <junie@jetbrains.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-02 11:47:09 -07:00
chrisfu
6d484ef13e docs(plans): k3d-mirror-of-GKE plan + Phase 1 brief for Junie
The next development thread is knoe-auth Phase 2 OIDC iteration, which
just landed on main but has no fast inner-loop. Instead of bouncing
edits through the GKE deploy chain, we want a laptop-resident model
where knoe-auth runs from the IDE / mvn against real Postgres + KDC
brought up in k3d.

Two new docs:

  docs/plans/k3d-gke-mirror.md
    Architectural plan. Top-to-bottom rationale for what the k3d
    model should mirror, what we deliberately skip (supabase, kong,
    oauth2-proxy, Studio — all unnecessary for knoe-auth's surfaces),
    and the phasing. Captures the three scope decisions taken
    2026-05-02 with the user:
      - knoe-auth runs on the laptop, not as a pod (faster inner loop)
      - stack scope is CNPG + KDC only
      - single-replica CNPG, realm KNOE.LOCAL (distinct from KNOE.DEV)
    Out-of-scope for Phase 1 listed explicitly so reviewer
    expectations match scope: SPNEGO E2E, image-build-and-load,
    supabase, oauth2-proxy, OidcCodeService DB persistence.

  docs/plans/junie/k3d-knoe-auth-dev-loop.md
    Phase 1 brief. Concrete deliverables:
      - k8s/knoe/knoe-kdc-{deployment,service,configmap,pvc,init-job}.yaml
      - etc/init_knoe_auth.sh --mode k3d flag (or sibling script)
      - Makefile: k3d-knoe-{up,pf,down}
      - etc/krb5.local.conf (with udp_preference_limit=1 for
        kubectl port-forward UDP flakiness on macOS)
      - docs/local-dev-knoe-auth.md (engineer-facing)
      - scripts/k3d-knoe-smoke.sh
    Definition of done: a fresh-clone laptop can `make k3d-knoe-up`
    + `make k3d-knoe-pf` + `mvn -pl authority spring-boot:run` and
    hit /health, /.well-known/openid-configuration in <8 minutes.

Index updates:
  docs/plans/README.md           — k3d-gke-mirror.md row added
  docs/plans/junie/README.md     — split into Active / Shipped;
                                   k3d brief listed under Active;
                                   the 5 shipped-2026-05-02 briefs
                                   moved to Shipped with commit refs.
  docs/TODO.md In-progress       — k3d brief now top of "In progress";
                                   the previously in-flight "Phase 2
                                   pg_oauth in install/deploy.sh"
                                   item demoted to "paused" (resume
                                   after the local dev loop lands).

No code changes; all docs.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-02 04:45:28 -07:00
chrisfu
232981c18c chore(cleanup): decommission garage on knoe-dev-0 app cluster (queue #4)
Mirroring the 2026-04-29 DB-cluster removal. Live state showed garage
in an unconfigured limbo (layout v0, no role assigned, 21d of "Ring
not yet ready" warnings); nothing outside garage's own state
referenced its S3 endpoint anymore.

Removed from knoe-system namespace on knoe-dev-0:
  - statefulset/garage      (1/1 pod, 21d uptime)
  - service/garage          (ClusterIP; no garage-s3-ilb on app cluster)
  - configmap/garage-config
  - secret/garage-secrets
  - pvc/data-garage-0       (29 Gi, garage-hdd)
  - pv/pvc-656e1936-...     (was Bound, reclaim=Retain)
  - storageclass/garage-hdd

Underlying GCE pd-standard disk (us-west3-c/pvc-656e1936-...) reclaimed
via `gcloud compute disks delete` since reclaim=Retain leaves the disk
behind after the PV is gone.

Source manifests intact:
  - k8s/knoe/garage-*.yaml                                   (k3d, min)
  - deploy/opentofu/k3s/manifests/knoe/garage-*.yaml         (k3s)
  - etc/init_garage_store.sh + knoe/core/ops/k8s_garage_store.py
  - k8s/knoe/garage-statefulset-gcp.yaml                     (GKE variant)

A future ./install.sh against GKE could re-deploy garage if that's
what the deploy mode wires; whether it SHOULD wire garage given GCS
now handles backups is a separate cleanup decision.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-02 03:58:47 -07:00
chrisfu
e52bbb8982 chore(cleanup): drop 5 stale resources; reframe queue #4 (garage)
Queue #5 Phase B orphan cleanup landed:

  App cluster (knoe-dev-0):
    - gitlab/gitlab-migrations-58a3e27-97-23-n72j8 (Failed/Evicted,
      11d stale)
    - pv/pvc-42e8545b-…  (10Gi, monitoring/storage-kps-grafana-0)
    - pv/pvc-86422179-…  (30Gi, monitoring/prometheus-…-prometheus-0)
    - pv/pvc-e2c29f7a-…  (5Gi,  monitoring/alertmanager-…-alertmanager-0)

  DB cluster (knoe-dev-cnpg-0):
    - kube-system/prole-supabase-dirprep Job (FailureTarget, 22d stale)

Post-state: 0 Released PVs on app cluster; gitlab namespace has only
the Completed migration pod; DB-cluster kube-system has no prole-*
artifacts. Live-cluster ops only; no source changes.

Queue #4 reframed: tried the planned `garage repair --yes blocks`,
but found garage in an unconfigured state — layout v0, no role
assigned, "Ring not yet ready" warnings continuous since whenever the
layout was lost (likely during a reconcile after the 2026-04-29 PVC
resize). Cross-cluster sweep shows nothing outside garage references
its S3 endpoint anymore (CNPG backups moved to GCS, no other workload
points at 10.180.15.239:3900). Garage on app-cluster is now a dead
pod holding an 80 GiB PVC. The original block-repair task is moot.
Reframed as a decommission-or-restore decision; details captured in
the ranked queue entry. Awaiting direction.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-02 03:50:51 -07:00
chrisfu
2ca2b8b234 docs(todo): note PodMonitor + DASHBOARD applied to live; node-label partial closure
Adds a 2026-05-02 Done entry covering the cluster-side work that paired
with c3fae73 (source) and 03bb731 (chart cleanups): PodMonitor migration
applied with verified zero-data-loss handoff; supabase chart upgraded
revision 1→3; helm field-manager conflict on Kong configmap resolved;
node label drift on knoe-dev-0 closed (3 nodes labelled
knoe.dev/node-role=general alongside legacy prole.org/node-role); DB
cluster nodes still pending same treatment.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-02 03:40:55 -07:00
chrisfu
03bb7310b4 fix(supabase): chart cleanups exposed during live helm upgrade
Two follow-ups to commit c3fae73 (queue #15) surfaced when applying
the DASHBOARD removal to live `supabase` release on knoe-dev-0:

1. **Delete `templates/test/kong.yaml`.** This was a Helm Test Job that
   validated the Studio dashboard's basic-auth gate. The basic-auth
   plugin was retired in commit 25f1b2e (oauth2-proxy fronting); the
   consumer + credentials in commit c3fae73. The test has nothing
   left to test — it referenced `supabase.secret.dashboard` template
   (gone) and `.Values.secret.dashboard.{secretRef,secretRefKey}`
   (gone), causing `helm template`/`upgrade` to fail. Removing the
   file is the right move; if dashboard auth ever comes back as
   something else, write a fresh test for that.

2. **Add `secret.oauth2proxy` + clarify `deployment.oauth2proxy`
   defaults in values.yaml.** The chart has
   `templates/studio/oauth2proxy-{deployment,service}.yaml` and
   `templates/secrets/oauth2proxy.yaml` (Junie's Phase 2 OIDC adds)
   guarded by `if .Values.deployment.oauth2proxy.enabled`. The guard
   short-circuits when the parent map exists, but if the map itself
   is undefined helm fails with `nil pointer evaluating interface{}.enabled`.

   Added a `secret.oauth2proxy` block (empty defaults) so the secrets
   template guard can evaluate cleanly. `deployment.oauth2proxy.enabled`
   was already `false` in values.yaml; added a comment explaining the
   chart oauth2-proxy templates are vestigial — the deployed
   oauth2-proxy lives in `deploy/gcp/gke/oauth2-proxy-deployment.yaml`,
   in FRONT of supabase-kong (commit 25f1b2e), not as a Kong upstream.

Live state after this commit:
  - `helm upgrade supabase ./supabase/helm/knoe-supabase
     --reset-then-reuse-values` succeeds.
  - revision 3 deployed; configmap field manager = `helm` (sole owner).
  - `kubectl get cm supabase-kong -o yaml | grep DASHBOARD` returns 0
    matches.
  - All Kong env vars on the new pod are clean (no DASHBOARD_*).
  - External smoke tests:
      /onboard.html → 200
      /rest/v1/ no auth → 401
      / (Studio gate) → 302 to Google
      /healthz → 302
      svc.knoe.dev/grafana/login → 200
  - cnpg-grafana panels still populate (PodMonitor migration verified
    earlier in same session: 3 postgres exporter targets up; cluster
    label injection working).

Side-finding handled live: nodes lacked the new
`knoe.dev/node-role=general` label that Junie's chart defaults to in
nodeAffinity. Added the label to all 3 knoe-dev-0 nodes alongside the
legacy `prole.org/node-role=general`. The label rename has been
tracked in CLAUDE.md as a follow-up; this is partial closure for the
app cluster (DB cluster nodes still need the same treatment when next
relevant).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-02 03:40:14 -07:00
chrisfu
c3fae73de3 fix(cnpg,kong): wire cnpg-backup-sa, migrate PodMonitor, drop DASHBOARD consumer
Three Junie briefs landed in one commit:

#07 — Wire cnpg-backup-sa into CNPG cluster spec (drift R8)
  deploy/gcp/gke/knoe-db.yaml: add spec.serviceAccountName: cnpg-backup-sa
  (requires CNPG v1.29+, which is the live operator version).
  etc/init_cnpg_gke.sh: operator install URL now uses CNPG_OPERATOR_VERSION
  variable (default 1.29.0); new §11 patches knoe-db and
  knoe-db-barman-cloud RoleBindings to add cnpg-backup-sa as a subject
  if not already present — matching the 2026-04-29 live stabilization.

#13 — Migrate off deprecated enablePodMonitor + podMonitorRelabelings
  Both deprecated fields removed from deploy/gcp/gke/knoe-db.yaml
  spec.monitoring. New deploy/gcp/gke/knoe-db-podmonitor.yaml carries
  the PodMonitor with the cluster relabeling rule (cnpg.io/cluster pod
  label → cluster label; required for all 85 CNPG Grafana panels).
  Apply alongside knoe-db.yaml on next cluster patch.

#15 — Remove dead DASHBOARD consumer + basicauth_credentials
  supabase/helm/knoe-supabase:
  - wrapper.sh: drop DASHBOARD_USERNAME / DASHBOARD_PASSWORD envsubst lines
  - config.yaml: drop DASHBOARD consumer + basicauth_credentials block
  - kong/deployment.yaml: drop both DASHBOARD env-var secret refs
  - values.yaml: rename secret.dashboard → secret.openai (apiKey only;
    username/password dropped — no enforcer since commit 25f1b2e)
  - secrets/dashboard.yaml + _helpers.tpl: renamed to openai /
    supabase.secret.openai
  - studio/deployment.yaml: reads from secret.openai.apiKey
  - ci/example.yaml: updated to secret.openai.apiKey
  helm template confirms knoe-supabase-openai secret referenced; no
  DASHBOARD output.

docs/TODO.md: queue items #7, #13, #15 + drift row R8 archived to Done.

Co-authored-by: Junie <junie@jetbrains.com>
2026-05-02 03:08:34 -07:00
chrisfu
5d17325c10 fix(scripts): patch_garage_cross_cluster — three defects from 2026-04-29 review
Defects A, B, C from docs/plans/junie/06-patch-garage-script-fixes.md:

  A. DB_CLUSTER default was knoe-cnpg-0 (stale rebrand artifact);
     corrected to knoe-dev-cnpg-0 to match conf/gke.cfg and the
     cluster table in CLAUDE.md. Header comment + overrides block
     also updated.

  B. Phase 1 deletion loop missed service/garage-s3-ilb (the
     LoadBalancer the ObjectStore endpoint historically pointed at).
     Now included alongside service/garage.

  C. Phase 2 was applying knoe-db-backup-gcs.yaml whole, including a
     legacy ScheduledBackup using method:barmanObjectStore (being
     removed in CNPG v1.30). Manifest split handled in prior commit
     (34a25dd); script's Phase 2 is now safe to re-run.

bash -n clean; CONFIRM=false dry-run prints knoe-dev-cnpg-0 correctly.
docs/TODO.md: queue items #2 and #6 + drift rows R5 and R9 archived to Done.

Closes queue item #6 in docs/TODO.md (drift R9).

Co-authored-by: Junie <junie@jetbrains.com>
2026-05-02 03:01:22 -07:00
chrisfu
34a25dde63 chore(deploy): split knoe-db-backup-gcs manifest — SA only, drop legacy ScheduledBackup
deploy/gcp/gke/knoe-db-backup-gcs.yaml previously contained both the
cnpg-backup-sa ServiceAccount (wanted) and a ScheduledBackup using
method: barmanObjectStore (unwanted — legacy in-tree method being
removed in CNPG v1.30, and the live cluster already has a plugin-method
ScheduledBackup that would be duplicated on re-apply).

Verified no other consumers of knoe-db-daily in the repo (grep clean).
File is now SA-only, matching its filename's intent.

Part 1/2 of queue item #6 fix (docs/plans/junie/06-patch-garage-script-fixes.md).

Co-authored-by: Junie <junie@jetbrains.com>
2026-05-02 03:00:51 -07:00
chrisfu
fb7e8b7135 chore(k3s): rename prole-*.yaml manifests to knoe-* and align contents
Mirrors commit b355855 (auth-side rename). The kustomization at
deploy/opentofu/k3s/manifests/knoe/kustomization.yaml has been
referencing knoe-* names since the rebrand; kustomize was broken
on the k3s deploy mode until this lands.

Five git mv:
  prole-configmap.yaml           -> knoe-configmap.yaml
  prole-deployment.yaml          -> knoe-deployment.yaml
  prole-service.yaml             -> knoe-service.yaml
  prole-kdc-configmap.yaml       -> knoe-kdc-configmap.yaml
  prole-kdc-secrets.example.yaml -> knoe-kdc-secrets.example.yaml

Plus content sweep of knoe-kdc-configmap.yaml: replaced stale
PROLE.LOCAL/PROLE.ORG realm refs and PROLE_KDC_* env vars with
KNOE.DEV/KNOE_KDC_* to match the canonical GKE configmap.

Closes queue item #2 in docs/TODO.md (drift R5).

Co-authored-by: Junie <junie@jetbrains.com>
2026-05-02 02:58:03 -07:00
chrisfu
00f0ebec07 Merge claude/crazy-bose-fec256 into main
Bringing the long-running session-feature branch back into main in one
deliberate sweep. The branch carried the cluster work that's been live for
weeks (cross-cluster CNPG metrics, Grafana w/ Google OAuth, supabase
oauth2-proxy, cluster recovery, pg.0.knoe.dev + per-engineer onboarding,
GCS-backed CNPG backups via Workload Identity, the env-contamination
guard, the Junie brief queue, the cnpg-grafana CSRF + memory-request
fixes from today), while main accumulated Junie's parallel knoe-auth
Phase 2 OIDC work (full provider surface: discovery, authorize, token,
userinfo, JWKS, RS256 signing, code exchange, session services).

Key decision: the two branches did COMPETING rebrands off the same
starting point (5ba9b63, 2026-04-27):

  - claude branch (commit b355855, earlier): org.prole.authority.* →
                                              dev.knoe.auth.*
                                              (artifact renamed to
                                              knoe-auth.jar)
  - main (commit 9daa94b, recent): org.prole.authority.* →
                                    dev.knoe.authority.*
                                    (kept "authority" artifact name)

dev.knoe.auth wins: cluster runs from this name, the Maven artifact is
already knoe-auth.jar, and the broader rename is the documented
namespace direction (per ~/.claude/projects/-Users-chrisfu-dev-knoe-db/
memory/MEMORY.md). All of main's recent Phase 2 OIDC content was ported
from authority/src/.../dev/knoe/authority/ into
authority/src/.../dev/knoe/auth/ with package declarations rewritten.

== File-level resolution summary ==

Textual conflicts (4):

  authority/pom.xml
    - Took our artifactId="auth"
    - Took our branch's removal of spring-security-kerberos-client
      (verified: Junie's Phase 2 OIDC code does not import it; the dep
      was already-dead config)

  docs/pipeline-phases.md
    - Took our branch's "Phase 1 not started" status. Main had a
      misplaced " Complete" with a knoe-auth-Phase-1 commit ref
      in the autobuild Phase 1 section — different domain.

  docs/plans/knoe-auth-round-1.md
    - Took our branch's dev.knoe.auth file table (vs main's
      dev.knoe.authority listing). Pure rename mismatch.

  supabase/helm/knoe-supabase/templates/kong/config.yaml
    - Took our branch's onboard route + plain dashboard wiring.
      Main had an oauth2proxy.enabled toggle that put oauth2-proxy as
      a Kong upstream — but the deployed architecture (commit 25f1b2e)
      has oauth2-proxy in FRONT of Kong, not behind. Main's wrapper
      reflected an architecture that was never deployed.
    - Took our branch's removal of basic-auth from dashboard route
      (queue #15 brief still tracks the matching values.yaml /
      kong/deployment.yaml cleanup).

Java tree reconciliation (44 file-pairs):

  20 dual-path source files + 2 dual-path tests
    Body-identical between main's authority/ and our branch's auth/
    after stripping package decls — main's commit 9daa94b was a pure
    rebrand. Took our branch's auth/ version for all 22.

  8 main-only source files (Phase 2 OIDC), ported into auth/:
    web/JwksController.java
    web/OidcAuthorizeController.java
    web/OidcDiscoveryController.java
    web/OidcTokenController.java
    web/OidcUserInfoController.java
    session/OidcCodeService.java
    session/OidcTokenService.java
    session/SessionService.java

  12 main-only test files, ported into auth/:
    HealthControllerTest.java
    enroll/EnrollValueTypesTest.java
    enroll/EnrollmentControllerTest.java
    enroll/TotpServiceTest.java
    kerberos/KadminClientTest.java
    kerberos/KerberosSpnegoResultTest.java
    web/LoginControllerTest.java
    admin/AdminControllerTest.java
    user/PrincipalNormalizerTest.java
    regression/IdentityRegressionTest.java
    session/OidcCodeServiceTest.java
    session/SessionServiceTest.java

  Port mechanics: read main:authority/...<file> via git show, then sed
  rewrite `package dev.knoe.authority` → `package dev.knoe.auth` and
  `import dev.knoe.authority` → `import dev.knoe.auth`. Body content
  unchanged.

  authority/src/main/java/dev/knoe/authority/ — DELETED (duplicate)
  authority/src/test/java/dev/knoe/authority/  — DELETED (duplicate)

== Verification ==

- grep -rln '<<<<<<<' across .java/.md/.yaml/.yml/.sh/.xml/.tpl: clean
- find authority/src -path '*/dev/knoe/authority*': empty (subtree gone)
- grep 'package dev.knoe.authority' across repo: clean
- bash -n install.sh deploy.sh etc/preflight_kubecontext.sh: clean
- git ls-files -u | wc -l: 0 unmerged paths
- helm lint supabase/helm/knoe-supabase: pre-existing failure on
  studioIngress.enabled undefined in values.yaml (introduced by Junie
  on main; unrelated to this merge — flagging as follow-up).

== Followups (carried into TODO ranked queue or noted here) ==

  - helm lint failure: studioIngress block in values.yaml is missing
    enable flag; templates/studio/{ingress,oauth2proxy-deployment,
    oauth2proxy-service}.yaml all reference studioIngress.enabled with
    no default. Pre-existing on main; not introduced by this merge.
  - The five Junie briefs filed on this branch are now reachable from
    main at docs/plans/junie/{02,06,07,13,15}-*.md. Junie can pick them
    up in any order.
  - knoe-auth Phase 2 OIDC source (now at dev.knoe.auth.*) is not yet
    deployed to the cluster. Deployment is its own task.
  - The branch claude/crazy-bose-fec256 stays in place (worktree at
    .claude/worktrees/crazy-bose-fec256 may have ongoing context for
    Claude Code sessions). Safe to delete once next session starts
    cleanly from main.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-01 16:39:10 -07:00
chrisfu
500c9b1317 fix(installer): env-contamination guard against shell-context / config mismatch
Filed in response to the 2026-04-28 14:00 UTC backup outage. An
`install.sh --mode k3d` run with the shell pointed at GKE silently
overwrote the GKE cluster's GCS-backed ObjectStore + ScheduledBackup
with k3d-mode defaults; Garage filled up and CNPG backups failed for
hours before the next manual check. The class of bug is "config says
target cluster A, shell context says target cluster B, installer
proceeds against B without warning."

New shared bash helper at etc/preflight_kubecontext.sh with two
functions:

  - verify_kubecontext_matches_config <cfg-path>
      Strict gate. Reads [Global] APP_CLUSTER_KUBECONTEXT from the
      config and exits 1 if `kubectl config current-context` differs.
      Skipped silently when the config has no baked APP_CLUSTER_KUBECONTEXT
      (e.g. fresh k3d.cfg) or when there's no live current-context.

  - print_kubecontext_notice
      Informational. Prints what's about to be inherited so the user
      can abort before the TUI launches if it looks wrong. Never fails.

Wiring:

  - deploy.sh sources the helper and calls the strict gate against
    ${PROLE_DEPLOY_CFG:-conf/gke.cfg} before invoking Python.
    Unattended path -> hard refusal on mismatch.

  - install.sh sources the helper and calls the informational notice
    (gated on not-`--min`) right after entering the local-checkout
    branch. The TUI is interactive, so the strict mode-aware gate is
    a follow-up once the welcome screen records a mode in
    state.inputs.

Bypass for deliberate cross-cluster maintenance:
    KNOE_SKIP_KUBECONTEXT_GUARD=true ./deploy.sh

End-to-end verified:
  - deploy.sh with current=cnpg-0, gke.cfg=app-0   -> exit 1, clear msg
  - deploy.sh with KNOE_SKIP_...=true              -> bypasses, prints
                                                     "skipping check"
  - install.sh --min                               -> notice skipped
  - install.sh (no flag) and install.sh --silent   -> notice printed

Doc updates:
  - CLAUDE.md §"Env-contamination warning" rewritten to describe the
    live guard (was a forward-looking TODO).
  - CLAUDE.md drift table row R4 removed; "Closed 2026-05-01" line added.
  - docs/TODO.md queue item #1 archived to Done; R4 dropped from the
    reality-vs-intent table. Queue numbering retained (no #1 placeholder)
    so the docs/plans/junie/<NN>-...md filenames still match.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-01 16:14:30 -07:00