Three independent lines of work landing in one commit because they were all
on disk together end-of-session and the cross-file edits (TODO, knoe-system,
junie/README) interleave cleanly:
1. **Queue #3 — image rename `knoe-authority` → `knoe-auth`** (Junie).
Closes drift R6. The Maven artifact has been `knoe-auth.jar` since
commit b355855; the deploy manifests now match.
authority/Dockerfile.app (NEW)
deploy/gcp/gke/knoe-auth-deployment.yaml (3 image tags renamed)
deploy/opentofu/k3s/manifests/knoe/knoe-auth-deployment.yaml (2 image tags renamed)
Makefile (docker-build-auth +
docker-push-auth + REGISTRY/KNOE_AUTH_VERSION
defaults)
docs/plans/junie/03-image-rename-knoe-authority-to-knoe-auth.md (brief, kept as design record)
2. **Phase 2 OIDC provider — GKE deploy** (Junie). Source landed via the
merge that brought claude/crazy-bose-fec256 back; the k3d sandbox
shipped earlier today (commit 93157b0). This commit completes the GKE
path: Kong route `/auth` → knoe-auth, default values added, signing
key wired via `knoe-auth-oidc` secret per the brief.
supabase/helm/knoe-supabase/templates/kong/config.yaml (Kong route /auth, gated
on .Values.knoeAuth.enabled)
supabase/helm/knoe-supabase/values.yaml (knoeAuth.enabled=false default;
studioIngress block — also
fixes the helm lint issue noted
in earlier commit 03bb731)
docs/plans/junie/phase2-oidc-gke-deploy.md (brief, kept as design record)
3. **k3d dev-user seed: `chrisfu` + `knoe_developer`** (Claude). Closes
"I want auth as chrisfu@knoey.com to my local cnpg database" for the
local dev loop. `etc/init_knoe_auth.sh --mode k3d` now runs
`seed_dev_users_k3d()` after the schema bootstrap, creating:
- `knoe_developer` group role with R/W on `knoe`+`public` (mirrors
the GKE production layout from docs/db-access.md; was hand-rolled
in production per the 2026-04-30 onboarding work, never baked into
postInitTemplateSQL).
- `chrisfu` LOGIN role with password `chrisfu-dev` (idempotent —
resets on every `make k3d-knoe-up`, so the rebuild loop is
deterministic). Granted into `knoe_developer`.
From the host with port-forward up:
PGPASSWORD=chrisfu-dev psql "postgresql://chrisfu@localhost:5432/knoe-db?sslmode=require"
Cross-cutting doc updates:
docs/TODO.md — Phase 2 GKE entry removed from "In progress"
(now "(none — all items shipped or paused)");
queue #3 + reality table R6 removed; pg_oauth
paused note flipped to "OIDC issuer now
deployed to GKE — pg_oauth can resume"; Done
section gets new entries for #3, Phase 2 GKE,
and chrisfu seed.
docs/knoe-system.md — Phase 2 GKE row → Shipped; pg_oauth → Ready;
knoe-auth-deployment.yaml note "knoe-auth:latest"
instead of "knoe-authority:latest"; "One-time
cluster setup" mentions the chrisfu seed.
docs/local-dev-knoe-auth.md — "Verify psql connectivity" rewritten to
connect as chrisfu (was `knoe`); new "Reset /
rebuild loop" section explaining idempotency.
docs/plans/junie/README.md — two new rows under Shipped: brief 03 and
phase2-oidc-gke-deploy.
Verification (the bits I ran locally):
- bash -n etc/init_knoe_auth.sh OK
- The seed SQL is idempotent (DO blocks with EXISTS checks +
ALTER ROLE on the password reset path)
- The Kong /auth route is gated on .Values.knoeAuth.enabled — default
false, no behavior change until someone flips it on per Junie's brief
Out of scope for this commit:
- Actually flipping knoeAuth.enabled=true on the live GKE chart and
rolling out — Junie's brief covers the runbook; needs the
knoe-auth-oidc K8s secret populated from 1Password first
- Rebuilding + pushing the new knoe-auth:latest image — `make
docker-push-auth` is wired, just hasn't been run yet
- Round 1.5 OpenBao transit-key encryption (still queued)
Co-authored-by: Junie <junie@jetbrains.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two follow-ups to commit c3fae73 (queue #15) surfaced when applying
the DASHBOARD removal to live `supabase` release on knoe-dev-0:
1. **Delete `templates/test/kong.yaml`.** This was a Helm Test Job that
validated the Studio dashboard's basic-auth gate. The basic-auth
plugin was retired in commit 25f1b2e (oauth2-proxy fronting); the
consumer + credentials in commit c3fae73. The test has nothing
left to test — it referenced `supabase.secret.dashboard` template
(gone) and `.Values.secret.dashboard.{secretRef,secretRefKey}`
(gone), causing `helm template`/`upgrade` to fail. Removing the
file is the right move; if dashboard auth ever comes back as
something else, write a fresh test for that.
2. **Add `secret.oauth2proxy` + clarify `deployment.oauth2proxy`
defaults in values.yaml.** The chart has
`templates/studio/oauth2proxy-{deployment,service}.yaml` and
`templates/secrets/oauth2proxy.yaml` (Junie's Phase 2 OIDC adds)
guarded by `if .Values.deployment.oauth2proxy.enabled`. The guard
short-circuits when the parent map exists, but if the map itself
is undefined helm fails with `nil pointer evaluating interface{}.enabled`.
Added a `secret.oauth2proxy` block (empty defaults) so the secrets
template guard can evaluate cleanly. `deployment.oauth2proxy.enabled`
was already `false` in values.yaml; added a comment explaining the
chart oauth2-proxy templates are vestigial — the deployed
oauth2-proxy lives in `deploy/gcp/gke/oauth2-proxy-deployment.yaml`,
in FRONT of supabase-kong (commit 25f1b2e), not as a Kong upstream.
Live state after this commit:
- `helm upgrade supabase ./supabase/helm/knoe-supabase
--reset-then-reuse-values` succeeds.
- revision 3 deployed; configmap field manager = `helm` (sole owner).
- `kubectl get cm supabase-kong -o yaml | grep DASHBOARD` returns 0
matches.
- All Kong env vars on the new pod are clean (no DASHBOARD_*).
- External smoke tests:
/onboard.html → 200
/rest/v1/ no auth → 401
/ (Studio gate) → 302 to Google
/healthz → 302
svc.knoe.dev/grafana/login → 200
- cnpg-grafana panels still populate (PodMonitor migration verified
earlier in same session: 3 postgres exporter targets up; cluster
label injection working).
Side-finding handled live: nodes lacked the new
`knoe.dev/node-role=general` label that Junie's chart defaults to in
nodeAffinity. Added the label to all 3 knoe-dev-0 nodes alongside the
legacy `prole.org/node-role=general`. The label rename has been
tracked in CLAUDE.md as a follow-up; this is partial closure for the
app cluster (DB cluster nodes still need the same treatment when next
relevant).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three Junie briefs landed in one commit:
#07 — Wire cnpg-backup-sa into CNPG cluster spec (drift R8)
deploy/gcp/gke/knoe-db.yaml: add spec.serviceAccountName: cnpg-backup-sa
(requires CNPG v1.29+, which is the live operator version).
etc/init_cnpg_gke.sh: operator install URL now uses CNPG_OPERATOR_VERSION
variable (default 1.29.0); new §11 patches knoe-db and
knoe-db-barman-cloud RoleBindings to add cnpg-backup-sa as a subject
if not already present — matching the 2026-04-29 live stabilization.
#13 — Migrate off deprecated enablePodMonitor + podMonitorRelabelings
Both deprecated fields removed from deploy/gcp/gke/knoe-db.yaml
spec.monitoring. New deploy/gcp/gke/knoe-db-podmonitor.yaml carries
the PodMonitor with the cluster relabeling rule (cnpg.io/cluster pod
label → cluster label; required for all 85 CNPG Grafana panels).
Apply alongside knoe-db.yaml on next cluster patch.
#15 — Remove dead DASHBOARD consumer + basicauth_credentials
supabase/helm/knoe-supabase:
- wrapper.sh: drop DASHBOARD_USERNAME / DASHBOARD_PASSWORD envsubst lines
- config.yaml: drop DASHBOARD consumer + basicauth_credentials block
- kong/deployment.yaml: drop both DASHBOARD env-var secret refs
- values.yaml: rename secret.dashboard → secret.openai (apiKey only;
username/password dropped — no enforcer since commit 25f1b2e)
- secrets/dashboard.yaml + _helpers.tpl: renamed to openai /
supabase.secret.openai
- studio/deployment.yaml: reads from secret.openai.apiKey
- ci/example.yaml: updated to secret.openai.apiKey
helm template confirms knoe-supabase-openai secret referenced; no
DASHBOARD output.
docs/TODO.md: queue items #7, #13, #15 + drift row R8 archived to Done.
Co-authored-by: Junie <junie@jetbrains.com>
Bringing the long-running session-feature branch back into main in one
deliberate sweep. The branch carried the cluster work that's been live for
weeks (cross-cluster CNPG metrics, Grafana w/ Google OAuth, supabase
oauth2-proxy, cluster recovery, pg.0.knoe.dev + per-engineer onboarding,
GCS-backed CNPG backups via Workload Identity, the env-contamination
guard, the Junie brief queue, the cnpg-grafana CSRF + memory-request
fixes from today), while main accumulated Junie's parallel knoe-auth
Phase 2 OIDC work (full provider surface: discovery, authorize, token,
userinfo, JWKS, RS256 signing, code exchange, session services).
Key decision: the two branches did COMPETING rebrands off the same
starting point (5ba9b63, 2026-04-27):
- claude branch (commit b355855, earlier): org.prole.authority.* →
dev.knoe.auth.*
(artifact renamed to
knoe-auth.jar)
- main (commit 9daa94b, recent): org.prole.authority.* →
dev.knoe.authority.*
(kept "authority" artifact name)
dev.knoe.auth wins: cluster runs from this name, the Maven artifact is
already knoe-auth.jar, and the broader rename is the documented
namespace direction (per ~/.claude/projects/-Users-chrisfu-dev-knoe-db/
memory/MEMORY.md). All of main's recent Phase 2 OIDC content was ported
from authority/src/.../dev/knoe/authority/ into
authority/src/.../dev/knoe/auth/ with package declarations rewritten.
== File-level resolution summary ==
Textual conflicts (4):
authority/pom.xml
- Took our artifactId="auth"
- Took our branch's removal of spring-security-kerberos-client
(verified: Junie's Phase 2 OIDC code does not import it; the dep
was already-dead config)
docs/pipeline-phases.md
- Took our branch's "Phase 1 not started" status. Main had a
misplaced "✅ Complete" with a knoe-auth-Phase-1 commit ref
in the autobuild Phase 1 section — different domain.
docs/plans/knoe-auth-round-1.md
- Took our branch's dev.knoe.auth file table (vs main's
dev.knoe.authority listing). Pure rename mismatch.
supabase/helm/knoe-supabase/templates/kong/config.yaml
- Took our branch's onboard route + plain dashboard wiring.
Main had an oauth2proxy.enabled toggle that put oauth2-proxy as
a Kong upstream — but the deployed architecture (commit 25f1b2e)
has oauth2-proxy in FRONT of Kong, not behind. Main's wrapper
reflected an architecture that was never deployed.
- Took our branch's removal of basic-auth from dashboard route
(queue #15 brief still tracks the matching values.yaml /
kong/deployment.yaml cleanup).
Java tree reconciliation (44 file-pairs):
20 dual-path source files + 2 dual-path tests
Body-identical between main's authority/ and our branch's auth/
after stripping package decls — main's commit 9daa94b was a pure
rebrand. Took our branch's auth/ version for all 22.
8 main-only source files (Phase 2 OIDC), ported into auth/:
web/JwksController.java
web/OidcAuthorizeController.java
web/OidcDiscoveryController.java
web/OidcTokenController.java
web/OidcUserInfoController.java
session/OidcCodeService.java
session/OidcTokenService.java
session/SessionService.java
12 main-only test files, ported into auth/:
HealthControllerTest.java
enroll/EnrollValueTypesTest.java
enroll/EnrollmentControllerTest.java
enroll/TotpServiceTest.java
kerberos/KadminClientTest.java
kerberos/KerberosSpnegoResultTest.java
web/LoginControllerTest.java
admin/AdminControllerTest.java
user/PrincipalNormalizerTest.java
regression/IdentityRegressionTest.java
session/OidcCodeServiceTest.java
session/SessionServiceTest.java
Port mechanics: read main:authority/...<file> via git show, then sed
rewrite `package dev.knoe.authority` → `package dev.knoe.auth` and
`import dev.knoe.authority` → `import dev.knoe.auth`. Body content
unchanged.
authority/src/main/java/dev/knoe/authority/ — DELETED (duplicate)
authority/src/test/java/dev/knoe/authority/ — DELETED (duplicate)
== Verification ==
- grep -rln '<<<<<<<' across .java/.md/.yaml/.yml/.sh/.xml/.tpl: clean
- find authority/src -path '*/dev/knoe/authority*': empty (subtree gone)
- grep 'package dev.knoe.authority' across repo: clean
- bash -n install.sh deploy.sh etc/preflight_kubecontext.sh: clean
- git ls-files -u | wc -l: 0 unmerged paths
- helm lint supabase/helm/knoe-supabase: pre-existing failure on
studioIngress.enabled undefined in values.yaml (introduced by Junie
on main; unrelated to this merge — flagging as follow-up).
== Followups (carried into TODO ranked queue or noted here) ==
- helm lint failure: studioIngress block in values.yaml is missing
enable flag; templates/studio/{ingress,oauth2proxy-deployment,
oauth2proxy-service}.yaml all reference studioIngress.enabled with
no default. Pre-existing on main; not introduced by this merge.
- The five Junie briefs filed on this branch are now reachable from
main at docs/plans/junie/{02,06,07,13,15}-*.md. Junie can pick them
up in any order.
- knoe-auth Phase 2 OIDC source (now at dev.knoe.auth.*) is not yet
deployed to the cluster. Deployment is its own task.
- The branch claude/crazy-bose-fec256 stays in place (worktree at
.claude/worktrees/crazy-bose-fec256 may have ongoing context for
Claude Code sessions). Safe to delete once next session starts
cleanly from main.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Kong pod crashlooped on startup with:
init_by_lua error: plugin 'request-termination' not enabled;
add it to the 'plugins' configuration property
The chart sets KONG_PLUGINS as an explicit allowlist, and the
/healthz route we added in commit e61e6a8 uses the request-termination
plugin (bundled with Kong 2.8 but not enabled unless listed). Append
it. No image change, no new dependency -- just the plugin's name in
the comma-separated list.
This should be the last blocker: all the infrastructure for a healthy
supabase-kong backend is already in place (BackendConfig, checksum
annotation on the pod template, merged ingress with both hosts +
both ManagedCerts, the .invalid URL fix). With Kong able to parse its
declarative config, the pod can enter Ready, GCE's /healthz probe
returns 200, and api.0.knoe.dev / db.0.knoe.dev both start serving.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Follow-up to 23f3f14. With the checksum annotation in place, `helm
upgrade` did start rolling Kong forward, but the new pod never became
Ready: `rollout status` timed out with
"1 old replicas are pending termination"
Kong's ConfigMap contains the /healthz route correctly, the BackendConfig
is HTTP /healthz, but the Kong process itself refuses to start cleanly
when one of its services has `url: http://127.0.0.1:8000/` -- Kong's own
proxy port. This self-reference apparently trips the declarative-config
parser (undocumented; symptomatic only).
Switch to a RFC-2606 `.invalid` placeholder. The request-termination
plugin short-circuits before any DNS lookup or upstream connection, so
a non-resolvable URL is functionally equivalent and Kong's parser has
nothing to complain about.
Same change in both places we define the /healthz route:
- supabase/helm/knoe-supabase/templates/kong/config.yaml (supabase-kong)
- etc/init_kong.sh inline kong.yml heredoc (knoe-svc-kong)
The knoe-svc-kong backend was already HEALTHY with the self-referential
URL -- different Kong instance, possibly different parser path -- but
keeping the two configs aligned so future edits don't drift.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Follow-up to 0f2fe93. Live post-deploy diagnostics showed that even
though helm upgrade rewrote the supabase-kong ConfigMap with the new
/healthz route, the existing Kong pod kept serving the old config: the
Deployment spec itself never changed, so no rolling restart happened.
Kong loads /usr/local/kong/kong.yml at startup and doesn't watch the
file, so the healthcheck probe against /healthz got 404 and the GCE
backend stayed UNHEALTHY. api.0.knoe.dev stayed broken.
Standard helm workaround: annotate the pod template with a sha256 of
the ConfigMap template. Any content change bumps the hash, which
changes the Deployment spec, which triggers a rolling restart. Pattern
matches what vector/deployment.yaml in this chart already does and is
widely used in the bitnami / ingress-nginx charts.
Studio has no ConfigMap volume mount (verified via grep), so no
equivalent annotation is needed there.
This lands the permanent B.1 fix from the plan. Whether the live Kong
pod needs a manual `kubectl rollout restart` to pick up the current
REVISION 2 config -- or whether the /healthz service URL itself is
tripping Kong's parser -- is still to be determined from Part A
diagnostics before any B.2 / B.3 edits.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Follow-up to 391c4f5. Live deploy showed GCE's L7 BackendConfig CRD
hard-rejects type: TCP with:
Error syncing to GCP: error running backend syncing routine:
error ensuring health check:
Protocol "TCP" is not valid, must be one of [HTTP,HTTPS,HTTP2]
The sync never completes, so the LB has no healthy backend and TCP
connections to the public endpoint just close (ERR_CONNECTION_CLOSED).
Fix: switch all three BackendConfigs to type: HTTP with request paths
that return 200:
- supabase-kong & knoe-svc-kong: add a dedicated /healthz route to the
Kong declarative config via the request-termination plugin, which
returns 200 synchronously with no upstream call. Equivalent liveness
semantics to the TCP check we wanted (backend is alive as long as Kong
accepts connections) but over HTTP, which GCE actually accepts.
- supabase/helm/knoe-supabase/templates/kong/config.yaml
- etc/init_kong.sh (inline kong.yml heredoc)
- supabase-studio: Studio returns 301 on / (Next.js default) so we
point the probe at /favicon.ico -- Next.js serves it as a static asset
with 200 unconditionally. Not as clean as a real readiness endpoint
but Studio does not expose one that returns 200 without auth.
- supabase/helm/knoe-supabase/templates/studio/backendconfig.yaml
Verified locally via helm template -f values.generated.json: the
rendered BackendConfigs come out with the HTTP protocol + correct paths,
and the Kong ConfigMap has the healthz service block before the
auth-v1-open service.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
GCE L7 Ingresses for knoe-svc-kong (svc.knoe.dev / api.knoe.dev),
supabase-kong (api.0.knoe.dev) and supabase-studio (db.0.knoe.dev) were
all stuck UNHEALTHY: the default GCE healthCheck is HTTP GET `/` on the
backend port, but Kong returns 404 on any unrouted path and Studio
returns a 301 redirect -- neither passes the default probe, so the LB
serves "Server Error" instead of reaching the pod.
Replicate the pattern already working for gitlab-webservice-default:
emit a BackendConfig CRD with a TCP healthCheck on the service port and
annotate the Service with cloud.google.com/backend-config so GCE picks
it up. TCP is sufficient for LB-level liveness -- the backend is "alive"
as long as the process is accepting connections.
- etc/init_kong.sh: new SVC_KNOE_BACKEND_CONFIG_NAME; apply BackendConfig
inside k8s/GCE branch; annotate Service post-apply.
- knoe-supabase chart: new kong/backendconfig.yaml + studio/backendconfig.yaml
(TCP on 8000 / 3000), gated on service.{kong,studio}.backendConfigName.
- knoe-supabase chart: kong/service.yaml + studio/service.yaml pick up
cloud.google.com/backend-config when backendConfigName is set.
- render_supabase.py: sets service.{kong,studio}.backendConfigName in k8s
mode so the above wires up automatically.
Separately, the chart-managed supabase-kong Ingress was being reaped
from the cluster seconds after helm install (manifest present in the
release, gone via `kubectl get`). Root cause TBD -- suspected
meta.helm.sh/* annotation ownership colliding with a GKE/Anthos audit
controller. Workaround: render_supabase.py now emits a standalone
public-ingress-kong.yaml (no helm metadata) that supabase/deploy.sh
applies alongside public-ingress-tls.yaml, and the chart template gains
an `ingress.externallyManaged` guard so it no-ops in k8s mode. Default
`false` keeps k3d/k3s behavior unchanged.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Introduced dependency validation for ManagedCertificate and FrontendConfig annotations in Supabase ingresses.
- Enhanced `deploy.sh` with public ingress TLS resource application and improved error diagnostics.
- Updated `render_supabase.py` to generate GKE TLS manifests and log rendered resources.
- Refined Supabase deployment flow to include detailed ingress dependency validation and reconciliation.
Summarize CLAUDE.md updates:
- Document both live GKE clusters as e2-standard-2 x3 in us-west3 and note stale app default in code.
- Call out DB cluster naming mismatch between conf (knoe-dev-cnpg-0) and code default (knoe-cnpg-0).
- Add SSD quota guidance: 300GB fully consumed by CNPG, so non-CNPG PVCs must use standard/pd-standard.
- Refresh reset script reference and quota section details to match current runtime state.
Co-authored-by: Junie <junie@jetbrains.com>
- Introduced `enforceGeneralNodeRole` to make node affinity and selector enforcement configurable.
- Updated `_split_frontdoor_docs` to handle custom release labels for Kong services.
- Enhanced Helm templates to support dynamic scheduling configurations.
- Added `reconcile_db_frontdoor_studio_pvcs` to migrate PVCs to the target storage class.
- Updated tests to validate custom release labels and scheduling logic.
- Added validation and explicit handling for Garage S3 endpoints in split APP/DB cluster setups.
- Improved dynamic role detection and authoritative context enforcement in initialization scripts.
- Updated Supabase and Garage configurations to support private cross-cluster endpoints, ensuring cluster interoperability and DNS constraints.
- Enhanced error handling and logging for invalid or missing configurations.
