prole/docs/TODO.md

Master TODO index

Single source of truth for unfinished work in this repo. Captures items that surfaced across the autobuild pipeline phases, the 2026-04-28 → 2026-04-29 cluster recovery / Maven build readiness work, the 2026-04-29 CNPG operator + backup stabilization work, and reality-vs-intent drift in the docs.

When you pick something up, move it from the ranked queue into the Now section. When it lands, archive it into the Done section at the bottom (or delete if it's not worth memorializing).

When you discover a new item, add it to the ranked queue. Default rank: bottom. Move it up later if it gets pulled in.

The Kanban "Now" section at top is the only place this doc imposes structure. Everything below "Now" is a numbered list — top of the list is next-most-important.

---

Now (Kanban)

In progress

• Phase 2: pg_oauth in install/deploy.sh + k3d build — assigned to Junie. The Phase 1 work (per-engineer roles, external LB at pg.0.knoe.dev, cert SANs, pg_hba tightening, docs/db-access.md) is committed. Phase 2 replaces the SCRAM password mechanism with PG18 native OAUTHBEARER, wires it into install.sh / supabase/deploy.sh so a fresh deploy lands with the OIDC mechanism active by default, and ports the Service + cluster cert pattern to k3d. Connection target stays at pg.0.knoe.dev; only the credential issuance changes (libpq Device Flow → Google Bearer token). Plan reference: ~/.claude/plans/we-re-continuing-work-on-happy-toast.md.

Up next

(empty — pull from the ranked queue below)

Blocked

(none currently)

---

Next (ranked queue — top of list = next-most-important)

Note: numbering retained from the original queue (no #1) so the Junie briefs in plans/junie/ keep their <NN>-… filename match. #1 (installer env-contamination guard) shipped 2026-05-01; see the Done section.

3. Image rename knoe-authority → knoe-auth — three deployment manifests still reference knoe-authority:latest:

   • deploy/gcp/gke/knoe-auth-deployment.yaml (3 lines)
   • deploy/opentofu/k3s/manifests/knoe/knoe-auth-deployment.yaml (2 lines) The Maven artifact is now knoe-auth.jar (post-rename in commit b355855). Image rebuild + registry push needed; both manifests then bump.

4. Garage block-resync corruption — after the 2026-04-29 bulk delete (8382 objects + 12 multipart aborts), garage-0 (on the app cluster knoe-dev-0 — not the DB cluster, that one was removed) background workers logged Could not fetch needed block X, no node returned valid data for orphaned block refs. Eventually self-heals via garage_block::resync worker, but worth verifying with garage repair --yes blocks once the dust settles.

5. Phase B orphan cleanup — leftover from the cluster recovery work that didn't get cleaned because we focused on CNPG/GCS:

   • App cluster (knoe-dev-0): delete gitlab/gitlab-migrations-58a3e27-97-23-n72j8 Error pod (7d+ stale); reap 3 Released PVs in monitoring/ (4h orphans from monitoring stack recreation)
   • DB cluster (knoe-dev-cnpg-0): delete kube-system/prole-supabase-dirprep Job (18d stale, prole-era)

6. Port oauth2-proxy manifest to k3d / k3s / min modes — deploy/gcp/gke/oauth2-proxy-deployment.yaml is GKE-shaped (uses cloud.google.com/v1 BackendConfig + GCE health-check semantics). For non-GCE deploy modes the same Deployment runs unchanged but the ingress glue differs (Traefik / nginx-ingress / no LB at all). Add a deploy/k3s/oauth2-proxy-deployment.yaml with the Traefik-flavored ingress annotations once we deploy that mode.

7. Wire GoTrue (supabase-auth) Google provider for Supabase API clients — different surface than the Studio gate (which oauth2-proxy now handles). Once we have client apps that authenticate end-users via Supabase Auth, configure GoTrue's Google provider with its own OAuth client. See docs/oidc-setup.md for the secret indirection pattern.

8. Audit-log story for the multi-tenant Studio phase — Studio doesn't track per-user actions in self-hosted; oauth2-proxy forwards the Authorization: Bearer <id_token> header to Kong (--pass-authorization-header=true / --set-authorization-header=true) but nothing reads it. When the multi-tenant control plane lands (per-user / per-project Studios), wire a Kong logging plugin that captures the email claim from the JWT and emits to a structured log sink.

9. Retire api.0.knoe.dev Ingress rule + managed cert — db.0.knoe.dev now serves both Studio (Google-gated) and the SDK API surface (Kong key-auth) via oauth2-proxy --skip-auth-route. api.0.knoe.dev remains as an alias for backward compatibility, but new docs/onboarding should point to db.0.knoe.dev. When external clients have all migrated: drop the api.0.knoe.dev rule from the supabase-kong Ingress and the supabase-api-managed-cert ManagedCertificate. Until then, the alias is harmless (same backend, same key-auth).

10. Fork the Supabase Studio image to wire in-app help / support / feedback buttons to mailto:support@knoe.dev (or https://db.0.knoe.dev/support) — upstream Studio (supabase/studio:2026.02.16-sha-26c615c) hardcodes those URLs to Supabase-cloud endpoints (supabase.com/dashboard/api/feedback etc.) which are unreachable from self-hosted, so the in-app "Report a problem" / "Send feedback" / "Get help" flows error out. No env-var hook exists in upstream — verified by Explore search. Fork the image, patch the relevant TSX (apps/studio/components/layouts/AppLayout/AppLayout.tsx, support-dialog component, settings/help links — handful of files), publish to our registry, bump image.studio.repository in supabase/helm/knoe-supabase/values.yaml. Same fork is the natural place to fix the OpenAI key panel, telemetry endpoints, and any other in-Studio assumptions about Supabase cloud as we encounter them. Stop-gap until then: https://db.0.knoe.dev/support 302s to mailto:support@knoe.dev (Kong route support); just need to tell users to bookmark or remember it.

11. Migrate cnpg-prometheus datasource UID to the stable cnpg-prometheus name — currently the live datasource has the auto-generated UID P5531627C358300FE from the original kps install. We pinned uid: cnpg-prometheus in monitoring/kps-values-gke.yaml so any FRESH kps install lands on the stable name, but Grafana refuses to change the UID of an already-provisioned datasource (read-only via API; rollout-restart doesn't migrate it). The dashboard transform monitoring/cnpg-dashboard-transforms.yaml ds_prometheus_default_to_cnpg carries the auto-uid as a workaround. Migration path on the next clean kps re-install (or after a maintenance window where we can wipe the Grafana sqlite DB to drop datasources): swap value: P5531627C358300FE → value: cnpg-prometheus in the transform and re-run the sync tool. No client-visible change either way.

---

Reality-vs-intent gaps

Items where docs describe an intended state that doesn't match live state. Each links to where reality is documented. Closing the gap means either updating the doc to match reality permanently, or doing the work to make reality match.

#	Reality (current)	Intended state	Documented at	Tracked work item
R6	Image refs knoe-authority:latest in deploy manifests	knoe-auth:latest (matches Maven artifact knoe-auth.jar post-rename)	Phase C plan follow-ups	Queue item #3

---

Later

11. Round 1.5: OpenBao transit-key encryption for per-user secrets at rest — authority/src/main/java/dev/knoe/auth/enroll/UserProvisioningService.java:84 has // TODO Round 1.5: encrypt secret with OpenBao transit key before storing. Currently TOTP secrets and similar long-lived per-user state go to knoe.* tables with PG-level encryption-at-rest only. Round 1.5 wraps writes/reads with an OpenBao transit-key envelope. See docs/plans/knoe-auth-round-1.md §8 "Out of scope for Round 1".

12. knoe-db/docker-entrypoint.sh:3 swap to set -Eeuo pipefail — current set -e only. Marked as TODO in the file: "swap to -Eeuo pipefail above (after handling all potentially-unset variables)". Needs an audit pass over the script for unset-var safety first.

13. Customer-deploy-resync execution decision — currently no separate ~/dev/prole working tree under development; the plan in docs/plans/customer-deploy-resync.md is dormant. Decision: do we ever activate the per-customer branching workflow (e.g. for a future second customer), or formalize "single-tenant for now" and remove the plan?

---

Deferred / Out of scope

• Studio Database Advisor — Performance: unindexed FKs in upstream-managed schemas — 6 FKs flagged on a fresh project: auth.mfa_challenges.factor_id, auth.saml_relay_states.flow_state_id, auth.oauth_authorizations.{client_id, user_id}, storage.s3_multipart_uploads_parts.{upload_id, bucket_id}. Adding indexes to upstream-managed tables risks conflict on the next supabase-auth / supabase-storage migration. Wait for upstream fixes; revisit if these turn out to be load-bearing for our usage.

• com.knoey.auth open-source split — saved for the major collaboration (per user memory: "we are saving knoey.com/auth for the major collaboration"). Will require a separate repo extraction and licensing pass.

• "Test ssh to Gitea" task — orphaned during a previous Cowork+Code session when the Dispatch connection was lost. May or may not still be relevant.

• Volume snapshot backups (CNPG VolumeSnapshot method) — alternative to barman tarball. Considered during the 2026-04-29 outage debugging when barman-cloud-backup looked broken; now that GCS+barman is verified working, no immediate need. Worth revisiting if barman shows further fragility.

• File the upstream plugin-barman-cloud issue about the misleading ReferenceError: weakly-referenced object no longer exists traceback that fires on persistent 5xx responses. Issue #707 is closed but didn't capture this pattern; ours is a clearer reproduction. Defer — the upstream is responsive but the workaround (sufficient bucket capacity + correct IAM) is good enough.

---

Done

(items get archived here when they land, with a date and commit reference; or just delete if not worth memorializing)

• 2026-05-02 — Wire cnpg-backup-sa into CNPG cluster spec + init script (queue #7 / drift R8). deploy/gcp/gke/knoe-db.yaml gains spec.serviceAccountName: cnpg-backup-sa (requires CNPG v1.29+). etc/init_cnpg_gke.sh: operator install URL now uses CNPG_OPERATOR_VERSION variable (default 1.29.0); new §11 patches both knoe-db and knoe-db-barman-cloud RoleBindings to add cnpg-backup-sa as a subject if not already present, matching the 2026-04-29 live stabilization. bash -n clean. Brief: docs/plans/junie/07-init-cnpg-gke-sa-wiring.md.

• 2026-05-02 — Migrate off CNPG-deprecated enablePodMonitor + podMonitorRelabelings (queue #13). Both deprecated fields removed from deploy/gcp/gke/knoe-db.yaml spec.monitoring. New sibling manifest deploy/gcp/gke/knoe-db-podmonitor.yaml carries the PodMonitor with the cluster relabeling rule (sourced from cnpg.io/cluster pod label; required for all 85 CNPG Grafana dashboard panels). Apply alongside knoe-db.yaml. Brief: docs/plans/junie/13-podmonitor-manual-management.md.

• 2026-05-02 — Remove dead DASHBOARD consumer + basicauth_credentials from Kong chart (queue #15). supabase/helm/knoe-supabase: removed DASHBOARD_USERNAME/DASHBOARD_PASSWORD envsubst lines from wrapper.sh, DASHBOARD consumer + basicauth_credentials block from config.yaml, and both env-var secret refs from kong/deployment.yaml. values.yaml secret.dashboard section renamed to secret.openai (only apiKey field; username/password dropped). templates/secrets/dashboard.yaml + _helpers.tpl updated to openai/supabase.secret.openai. studio/deployment.yaml updated to read from secret.openai.apiKey. ci/example.yaml updated. helm template confirms knoe-supabase-openai secret referenced, no DASHBOARD output. Brief: docs/plans/junie/15-remove-dead-dashboard-consumer.md.

• 2026-05-02 — k3s manifest rename: prole-* → knoe-* (queue #2 / drift R5) in commit fb7e8b7. Five git mv in deploy/opentofu/k3s/manifests/knoe/: prole-{configmap,deployment,service,kdc-configmap,kdc-secrets.example}.yaml → knoe-*. Content sweep of knoe-kdc-configmap.yaml: replaced stale PROLE.LOCAL/PROLE.ORG realm refs and PROLE_KDC_* env vars with KNOE.DEV/KNOE_KDC_* to match the canonical GKE configmap. Kustomize renders cleanly (kubectl kustomize → 1193 lines, exit 0). Brief: docs/plans/junie/02-k3s-prole-rename.md.

• 2026-05-02 — Fix scripts/patch_garage_cross_cluster.sh (queue #6 / drift R9) in commits 34a25dd + e9df3c4. Three defects from the 2026-04-29 one-shot run: (A) DB_CLUSTER default knoe-cnpg-0 → knoe-dev-cnpg-0 (stale rebrand artifact; header comment + overrides block also updated); (B) Phase 1 deletion loop now includes service/garage-s3-ilb (the LoadBalancer that was left orphaned after the original run); (C) deploy/gcp/gke/knoe-db-backup-gcs.yaml split — legacy ScheduledBackup with deprecated method: barmanObjectStore removed, file is now SA-only. bash -n clean; CONFIRM=false dry-run prints correct cluster names. Brief: docs/plans/junie/06-patch-garage-script-fixes.md.

• 2026-05-01 — Installer env-contamination guard (queue #1 / drift R4) (commit pending). Filed in response to the 2026-04-28 14:00 UTC backup outage (an install.sh --mode k3d run with the shell pointed at GKE replaced the GCS-backed ObjectStore with a Garage-backed one). New shared bash helper etc/preflight_kubecontext.sh with two functions: verify_kubecontext_matches_config <cfg> (strict gate; reads [Global] APP_CLUSTER_KUBECONTEXT from the config and exits 1 if kubectl config current-context differs), and print_kubecontext_notice (informational). deploy.sh sources the helper and calls the strict gate before invoking the Python deploy pipeline. install.sh calls the informational notice (gated on not---min) so the user sees the inherited context up-front before the TUI launches; the mode-aware strict gate for the interactive path is a follow-up for the welcome screen once it records mode. Bypass with KNOE_SKIP_KUBECONTEXT_GUARD=true for deliberate cross-cluster maintenance. Verified end-to-end: deploy.sh refuses on mismatch with a clear remediation message; bypass override works; install.sh's --min path skips the notice; install.sh's no-flag and other-flag paths print the notice. CLAUDE.md drift table row R4 removed; §"Env-contamination warning" rewritten to describe the live guard. Closes the 2026-04-28 14:00 UTC outage feedback loop.

• 2026-05-01 — Cross-cluster CNPG metrics via DB-cluster Prometheus (commit pending). Solves the cloudnative-pg Grafana dashboard's "No data" problem from queue item #12. Picked path (c) over the originally-recommended (b) for DB-cluster autonomy + pattern symmetry — each cluster owns its observability footprint, and a future second DB cluster reuses the recipe one-for-one. Installed kube-prometheus-stack v84.3.0 on knoe-dev-cnpg-0 (monitoring namespace), Grafana + Alertmanager disabled (single canonical Grafana stays on app cluster). All PVCs explicitly pinned to standard-hdd (per the user's standing rule on SSD-quota preservation; verified prometheus-...-0 PVC bound 20Gi standard-hdd). New ILB prometheus-cnpg-ilb in DB-cluster monitoring ns at 10.180.15.216:9090, reachable from app-cluster pods over the shared VPC. New cnpg-prometheus datasource added to app-cluster Grafana via monitoring/kps-values-gke.yaml grafana.additionalDataSources. Side-fix: removed an orphan knoe-grafana-datasource ConfigMap that etc/init_monitoring.sh:apply_grafana_datasource() had been creating with a duplicate uid: prometheus — caused Grafana's provisioning reload to fail with HTTP 500 once any second datasource arrived. Function neutered to a no-op with a clear comment; chart values are now the single source of truth for datasources. Verified end-to-end: Prometheus on DB cluster discovers all 3 CNPG postgres pods (knoe-db-2/3/4 at 10.24.x.x:9187, all health=up); Grafana datasource health endpoint returns Successfully queried the Prometheus API; cnpg_pg_stat_archiver_archived_count query through Grafana proxy returns live data (704 archived WAL files at time of test). To complete: switch each cnpg-grafana dashboard's DS_PROMETHEUS template variable to cnpg-prometheus (one-click). New files: monitoring/kps-cnpg-values.yaml, deploy/gcp/gke/cnpg-prometheus-ilb.yaml. Modified: monitoring/kps-values-gke.yaml (datasource added), etc/init_monitoring.sh (function no-op'd). Out of scope (Phase 2): federate alerts back to app-cluster Alertmanager; reserve a static internal IP so the ILB doesn't drift on re-create.

• 2026-04-30 — Bump postgres pod memory limit 512Mi → 2Gi (commit pending). Long-standing throughput cliff: barman-cloud-backup is single-threaded gzip + GCS upload buffering, ceiling-bound by the pod's memory limit. The 9 GB DB took 30–90 min to back up at 512Mi; 2Gi cuts that to 5–10 min. requests left at 128Mi (no scheduling change; this is purely a ceiling bump). Patched live with kubectl patch cluster knoe-db --type=merge -p '{"spec":{"resources":{"limits":{"memory":"2Gi"}}}}'; CNPG operator performed a supervised rolling restart (replicas first, primary last). Repo source updated at deploy/gcp/gke/knoe-db.yaml. Pre-flight verified no external psql sessions (chrisfu/ron not connected); 39 internal sessions FATAL'd through the rolling restart and reconnected via their pools — supabase + gitlab + streaming_replica all stable post-rollout. Closes drift R3. k3s + k8s mode resource limits remain at 512Mi (not exposed in those manifests today); cross-mode parity folded into queue item #8 ("Port oauth2-proxy manifest to k3d/k3s/min modes").

• 2026-04-30 — Align knoe/core/actions.py cluster-name + machine-type defaults with live state (commit pending). Two stale constants since the rebrand: DEFAULT_DB_CLUSTER_NAME = "knoe-cnpg-0" (live: knoe-dev-cnpg-0) and DEFAULT_APP_CLUSTER_MACHINE_TYPE = "e2-small" (live: e2-standard-2). Updated both at knoe/core/actions.py lines 121 + 123. Verified: python3 -c "from knoe.core.actions import ..." reads the new values; pytest collection error count went from 8 → 7 post-edit (one of the pre-existing collection errors apparently went away when the import resolved differently — no regression). Closes drift R1 + R2 in CLAUDE.md (footnotes + drift table rows + the # STALE inline comments in the §"Cluster code constants" code block all dropped). Implicitly closes the formerly-#8 queue item ("Update CLAUDE.md cluster table once defaults match") in this same commit. Queue renumbered: #2 (this item) and #8 dropped, gaps closed.

• 2026-04-30 — Drop redundant GCP IAM workloadIdentityUser binding (commit pending). v1.28-era leftover: [knoe-db-0/knoe-db] was bound alongside the active [knoe-db-0/cnpg-backup-sa] on the cnpg-backup@… GCP SA. Pods have used cnpg-backup-sa since the 2026-04-29 stabilization (commit e92cd8b), so the older binding was dead config — kept around because removing it required a reasoned check that no in-flight job still authenticated under the old identity. Verified live (gcloud iam service-accounts get-iam-policy), dropped via gcloud iam service-accounts remove-iam-policy-binding, re-verified the SA's policy now has only [knoe-db-0/cnpg-backup-sa]. CNPG primary still healthy (SELECT 1 returned cleanly post-removal). Closes drift R7. Removed the matching CLAUDE.md Reality TODO + drift table row.

• 2026-04-30 — Grafana at svc.knoe.dev/grafana with Google OAuth (commit pending). Diagnosed the svc.knoe.dev "name resolution failed" error: knoe-svc-kong-config had its grafana service pointing at prometheus-grafana.monitoring.svc.cluster.local:80 — the OLD upstream name from before the kube-prometheus-stack chart renamed to kps-grafana. Kong was DNS-failing every request to that vanished service and surfacing the Kong-style "name resolution failed" error to clients. Fixed the upstream + scoped the route to /grafana (so other surfaces can land at svc.knoe.dev/* later). Layered Grafana's native Google OAuth via auth.google config in monitoring/kps-values-gke.yaml (new — captures previously-uncommitted live Helm values + adds the auth additions). Role mapping via JMESPath: contains(['chrisfu@knoey.com', 'ron@knoey.com'], email) && 'Admin' || 'Editor' — anyone @knoey.com gets Editor, chrisfu+ron get Admin. New manifests: deploy/gcp/gke/grafana-google-oidc-secret.example.yaml, etc/init_grafana_oauth.sh, monitoring/kps-values-gke.yaml. Repo Kong source updated at deploy/opentofu/k3s/manifests/knoe/kong-configmap.yaml (.org→.dev rebrand, healthz route restored, /grafana scoping). Verified externally: /grafana/login → 200; /grafana/login/google → 302 accounts.google.com with the right client_id; subpath assets load; Studio + SDK paths on db.0.knoe.dev unchanged. Browser walkthrough (chrisfu sign-in → Admin role; ron same; outside-domain rejected by allowed_domains: knoey.com) is the manual follow-up. Stop-gap adminPassword: admin in values stays (local break-glass) — rotate this when convenient as a follow-up.

• 2026-04-30 — Reusable engineer onboarding via one-time link + QR (commit pending). Replaces the assumed-but-nonexistent shared 1Password vault with a self-contained delivery flow: ./etc/onboard_engineer.sh <user> <email> provisions the postgres role + 24h temp password and outputs (1) a single-shot URL, (2) a QR-code rendering for screenshare, (3) the plaintext fallback. Engineer opens URL or scans QR → static page at db.0.knoe.dev/onboard.html (vanilla HTML+JS, no backend) reads the URL fragment client-side and shows password + psql connection string + rotation snippet with [Copy] buttons; saves to engineer's personal 1Password. Engineer rotates immediately via \password. New manifests: deploy/gcp/gke/knoe-onboard.yaml (nginx:alpine + ConfigMap + Service), Kong route /onboard.html → knoe-onboard:80 in supabase/helm/knoe-supabase/templates/kong/config.yaml, oauth2-proxy --skip-auth-route=^/onboard\.html$ in deploy/gcp/gke/oauth2-proxy-deployment.yaml. Pivotal pg_hba change: rule changed from hostssl all chrisfu,ron all to hostssl all +knoe_developer all so future onboards don't require pg_hba edits — the script's GRANT knoe_developer TO <user> is enough. CA cert committed at etc/knoe-db-ca.crt (public root, not a secret) so engineers reference sslrootcert=etc/knoe-db-ca.crt after cloning. Engineer-facing docs: docs/onboarding.md. docs/db-access.md updated to remove the broken 1Password reference. Verified end-to-end: ./etc/onboard_engineer.sh test-onboard test-onboard@knoey.com → URL + QR rendered → psql ...user=test-onboard... connects with the temp password and reports pg_has_role(current_user,'knoe_developer','MEMBER')=t → --revoke drops the role cleanly.

• 2026-04-30 — Per-engineer psql access via pg.0.knoe.dev (Phase 1) (commit pending). External GKE TCP LoadBalancer at 34.106.156.196 (forwarding rule a61028024f2314e06902bafc1b4c3a08); user adds Namecheap A record pg.0.knoe.dev → 34.106.156.196 (manual; Namecheap UI). CNPG serverAltDNSNames extended with pg.0.knoe.dev so the auto-issued server cert validates externally. New knoe_developer group role with R/W on knoe+public and R/O on auth+storage+extensions. Per-engineer roles chrisfu and ron with strong random passwords (192-bit base64) VALID UNTIL '2026-05-30', members of knoe_developer. pg_hba rewritten so external requires TLS+SCRAM and only allows chrisfu, ron (other roles postgres, supabase_admin, authenticator stay cluster-internal). externalTrafficPolicy: Local on the LB to preserve client source IP (otherwise the SNAT'd node IP made externals look like internal cluster traffic and bypassed the hostnossl reject rule). Documentation in docs/db-access.md. Verified: chrisfu+ron both connect via TLS, plaintext rejected with the proper pg_hba error showing the real client IP, postgres role connection from outside fails auth, internal supabase services keep their existing TLS connections (gitlab on TLS 1.3, supabase_admin/authenticator on plaintext over RFC1918 — both still allowed by the new rules). Phase 2 (libpq OAUTHBEARER + validator library, bundled into install/deploy.sh + k3d) handed off to Junie — see "In progress" at top of file.

• 2026-04-30 — pg_stat_statements moved out of public schema across deploy modes (commit pending). Studio's Database Advisor was flagging public.pg_stat_statements as a Security warning the moment any user opened the dashboard. Root cause: relocatable extensions without an explicit SCHEMA clause on CREATE EXTENSION land in the first writable schema in the connecting role's search_path, which is public for the postgres role. Live: ALTER EXTENSION pg_stat_statements SET SCHEMA extensions;. Bootstrap (3 files, all 4 deploy modes): deploy/gcp/gke/knoe-db.yaml (gke), deploy/opentofu/k3s/manifests/knoe/knoe-db.yaml (k3s), k8s/knoe/knoe-db.yaml (k3d / min) — each postInitTemplateSQL block now creates the extensions schema and CREATE EXTENSION ... SCHEMA extensions;. Plus supabase/deploy.sh gets GRANT USAGE ON SCHEMA extensions TO anon, authenticated, service_role; so PostgREST clients (and the advisor itself) can resolve the schema. Other security checks verified clean (no tables in public without RLS, no SECURITY DEFINER without search_path, no auth.users-exposed views, no duplicate indexes, no extensions left in public). Performance side has 6 unindexed FKs in upstream auth.* and storage.* tables — captured as deferred (touching upstream-managed schemas is the wrong move; they'll get fixed in supabase-auth / supabase-storage releases).

• 2026-04-30 — Studio post-launch fixes (storage permissions + /support stop-gap) in commit 79a3f76:

  • Storage list-buckets unblocked. supabase_storage_admin role was missing membership in service_role / authenticated / anon, so SET LOCAL role = 'service_role' (run by supabase-storage on every request) failed with PG 42501 at guc.c:6936 call_string_check_hook, surfaced as a misleading "new row violates row-level security policy" error. One GRANT applied live (GRANT anon, authenticated, service_role TO supabase_storage_admin;) and added durably to supabase/deploy.sh right after the existing GRANT service_role TO authenticator block.
  • oauth2-proxy auth-header passthrough fixed. The --pass-authorization-header=true and --set-authorization-header=true flags were stomping the Authorization: Bearer <apikey> header that Studio / supabase-py send on /storage/v1/... calls (oauth2-proxy was rewriting it with the Google OIDC id_token, which storage couldn't validate against JWT_SECRET). Both flags removed from deploy/gcp/gke/oauth2-proxy-deployment.yaml. When per-user identity passthrough lands post-knoe-auth Round 1, use --pass-user-headers / --set-xauthrequest (X-Forwarded-* headers) instead — those don't collide with Supabase's auth.
  • /support route stop-gap. https://db.0.knoe.dev/support 302s to mailto:support@knoe.dev so users have a working escape hatch while the in-Studio support buttons stay broken (separate Studio-fork follow-up #15). Kong service support added to templates/kong/config.yaml using request-termination (302) + response-transformer (Location header). KONG_PLUGINS allowlist extended with response-transformer in values.yaml. oauth2-proxy gets --skip-auth-route=^/support$ so an unauthenticated user looking for help isn't bounced into a Google sign-in detour.

• 2026-04-30 — Unified db.0.knoe.dev for Studio + SDK in commit f805404: added two --skip-auth-route args to oauth2-proxy so SDK requests on /auth/v1, /rest/v1, /realtime/v1, /storage/v1, /functions/v1, /graphql/v1, /pg/* bypass the Google sign-in flow and pass through to supabase-kong, where Kong's existing key-auth plugin enforces anon/service-role keys. Studio UI at / remains Google-gated. One URL now covers humans-in-Studio AND robots-via-supabase-py — same security envelope as api.0.knoe.dev had on those paths. api.0.knoe.dev kept as a backward-compat alias (see follow-up #14). External verification: curl -H "apikey: <KEY>" https://db.0.knoe.dev/rest/v1/ → 200 OpenAPI; curl https://db.0.knoe.dev/rest/v1/ (no key) → 401 from Kong; curl https://db.0.knoe.dev/ → 302 to Google (Studio gate intact); curl https://db.0.knoe.dev/api/profile → 302 to Google (Studio internals stay gated).

• 2026-04-30 — Google sign-in to db.0.knoe.dev for chrisfu + ron (shared Studio) in commit 25f1b2e: oauth2-proxy v7.15.2 deployed in front of supabase-kong as the Studio access gate (Google Workspace OIDC, --email-domain=knoey.com); supabase-kong Ingress patched so db.0.knoe.dev → oauth2-proxy:80 while api.0.knoe.dev → supabase-kong:8000 is untouched (key-auth still gates the API surface); Kong dashboard route's basic-auth plugin removed (oauth2-proxy is the gate now — double-auth was friction without security gain). External verification: db.0.knoe.dev/ → 302 accounts.google.com/o/oauth2/auth?client_id=200699021716-…&redirect_uri=https://db.0.knoe.dev/oauth2/callback&scope=openid+email+profile; api.0.knoe.dev/rest/v1/ → 401 from Kong key-auth (untouched); /healthz → 200. Three new files + chart edit: deploy/gcp/gke/oauth2-proxy-deployment.yaml, deploy/gcp/gke/oauth2-proxy-google-oidc-secret.example.yaml, etc/init_oauth2_proxy.sh, and the basic-auth plugin removed from supabase/helm/knoe-supabase/templates/kong/config.yaml. Future swap to knoe-auth Round 1: change one arg in oauth2-proxy from --provider=google to --provider=oidc --oidc-issuer-url=https://api.knoe.dev/auth and reapply.

• 2026-04-29 — CNPG operator + backup stabilization session (4 queue items in one wave; live-cluster ops, no commit yet — doc-update commit pending):

  • #4 done: plugin-barman-cloud operator restarted (kubectl rollout restart deployment/barman-cloud -n cnpg-system); stale Garage retention-worker endpoint cache cleared
  • #11 done: cnpg-controller-manager restarted; clean plugin re-discovery confirmed in logs
  • CNPG operator upgraded v1.28.1 → v1.29.0 to expose spec.serviceAccountName field (added in v1.29.0 per upstream release notes); operator rollout clean, cluster healthy throughout
  • #2 done: cluster.spec.serviceAccountName=cnpg-backup-sa applied; supervised rolling restart cycled all 3 pods (db-3 first failed with RBAC error — patched RoleBindings knoe-db and knoe-db-barman-cloud to add cnpg-backup-sa as a subject; db-3 then recovered, db-4 and db-2 rolled cleanly); 23:03 UTC scheduled backup completed with new SA, verifying end-to-end
  • #3 done: Garage workload removed from knoe-dev-cnpg-0 (statefulset, both services including garage-s3-ilb, configmap, PVC data-garage-0, StorageClass garage-hdd); ObjectStore untouched, backups still progressing post-removal

• 2026-04-29 — Doc true-up + master TODO index in commit f5b24c7. Reality-vs-intent gaps captured across CLAUDE.md / AGENTS.md / pipeline-phases / plans / canonical ObjectStore manifest; new docs/TODO.md Kanban-style index.

• 2026-04-29 — GCS migration completed. ObjectStore gs://knoe-0-backups/ with googleCredentials.gkeEnvironment: true; WI auth via knoe-db SA. 2 verified backups in catalog. ScheduledBackup unsuspended. (live-cluster ops; no commit)

• 2026-04-29 — Garage cluster recovered. PVC 29 → 80 GiB, layout 10 → 60 GB, bucket cleared (8382 objects + 12 orphan multiparts). (live-cluster ops; no commit)

• 2026-04-28/29 — knoe-db-1 fix. 14d crashloop resolved via PVC delete + CNPG re-bootstrap; cluster failover-cycled all 3 pods; 3/3 healthy. (live-cluster ops; no commit)

• 2026-04-28 — Phase A repo cleanup + Phase C Maven build readiness in commit b355855. tmp/ archived (87 files); org.knoe → dev.knoe rename; authority → auth artifactId; sources moved to dev/knoe/auth/; make build-auth green. 4 k3s prole-auth-*.yaml manifests renamed to knoe-auth-*.yaml.

• 2026-04-28 — Pipeline Phase 0 (test pipeline foundation: pyproject.toml, IntelliJ run configs, coverage fix, welcome mode selector). See docs/pipeline-phases.md §Phase 0.