Commit Graph

87 Commits

Author SHA1 Message Date
chrisfu
c9591a5da2 docs(runbook): stage db.prole.org GSSAPI auth for the airgap lane
Complete, review-ready runbook to light up Kerberos/GSSAPI on db.prole.org so
`kn db --service prole.org` authenticates with a local PROLE.ORG ticket — the
airgap data lane (no password, no knoe-auth, no internet).

Nothing applied. Delivers:
- Samba SPN + keytab export steps (myrddin): postgres/db.prole.org@PROLE.ORG
- k8s keytab secret (knoe-db-gss-keytab-secret.example.yaml template)
- exact knoe-db.yaml diff: krb_server_keyfile, hostgssenc pg_hba (include_realm=0
  → role), projectedVolumeTemplate keytab mount, chrisfu/ron managed roles
- apply + end-to-end verify (kinit → kn db --service prole.org)
- failure-mode triage, rollback, follow-ups

Hooks into the manifest's existing placeholders (knoe-db.yaml lines 38 + 45,
which already note 'Kerberos disabled on k3s'). Image is already --with-gssapi.

Flags the PROLE.ORG (Kerberos) vs PROLE.LOCAL (knoe.user default) discrepancy
to reconcile. Grants intentionally deferred — connect succeeds, reads gated.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-03 02:59:07 -07:00
chrisfu
d114801758 feat: wire ekosystem UUID system into CNPG manifests (Tasks 1, 3, 4)
Task 1 — ConfigMap + CNPG wiring:
- Add k8s/knoe/knoe-ekosystem-sql.yaml: ConfigMap embedding ekosystem.sql
  and ekosystem_objects.sql for CNPG postInitApplicationSQLRefs
- Add scripts/gen-ekosystem-configmap.py: generation script to keep the
  ConfigMap in sync with knoe-db/schema/ekosystem*.sql source files
- Add Makefile target: make k8s/knoe/knoe-ekosystem-sql.yaml
- Wire postInitApplicationSQLRefs into all three CNPG cluster manifests:
    k8s/knoe/knoe-db.yaml (k3s / prole-service-context production)
    deploy/gcp/gke/knoe-db.yaml (GKE)
    deploy/opentofu/k3s/manifests/knoe/knoe-db.yaml (OpenTofu k3s)
- Add knoe-ekosystem-sql.yaml to k8s/knoe/kustomization.yaml

Task 3 — Python counterpart utility:
- Add knoe/ekosystem.py: thread-safe EkosystemID generator matching the
  PostgreSQL bit layout [49:ts_ms|12:tenant|10:shard|11:seq], with
  decode() and can_access() helpers
- Add tests/test_ekosystem.py: 23 tests covering base36 encoding,
  round-trips, thread safety, can_access, and the spec round-trip assertion

Task 4 — knoe.user ekosystem_uuid column:
- Add ALTER TABLE knoe.user ADD COLUMN IF NOT EXISTS ekosystem_uuid text UNIQUE
  to postInitSQL in all three CNPG manifests

Task 2 (register prole tenant) requires a live DB connection — manual step.
Task 5 (LDAP/AD reconciler) is design-only per spec.

Co-authored-by: Junie <junie@jetbrains.com>
2026-05-30 00:26:48 -07:00
chrisfu
dc099b99f9 fix(knoe-auth): wire Google Workspace login for prole.org
- Add KNOE_AUTH_BASE_URL=https://api.prole.org/auth to knoe-auth
  deployment; LoginController.java defaulted to api.0.knoe.dev causing
  redirect_uri mismatch on every Google login attempt

- Update oauth2-proxy-deployment-prole.yaml to reflect live config:
  --provider=oidc --oidc-issuer-url=https://api.prole.org/auth (was
  stale --provider=google; live deployment already switched to OIDC)

- Fix redirect URIs in knoe-auth-google-prole-secret.example.yaml:
  remove spurious double /auth/auth/ prefix (Kong strip_path=false so
  Spring @RequestMapping(/auth) gives /auth/callback/google not
  /auth/auth/callback/google)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-28 07:11:13 -07:00
chrisfu
c1d2a91991 fix(spnego): Subject.callAs, AES keytab, and krb5.conf sync for JDK 21
- KerberosSpnegoService: replace Subject.doAs with Subject.callAs (JDK 18+)
  so Subject.current() is set correctly for JGSS credential lookup in JDK 21.
  Pass null GSSCredential to createContext so the mechanism auto-selects
  rather than failing with "No credential found for 1.3.6.1.5.5.2".
- knoe-auth-deployment: fix PROLE_KERBEROS_KEYTAB_PATH to /etc/knoe-auth/http.keytab
  (matches the knoe-auth-http-keytab secret volume mount).
- knoe-kdc-configmap: full rewrite from stale KNOE.DEV to KNOE.LOCAL;
  add PROLE.ORG realm (Samba AD at 10.0.0.3) and [capaths] for future
  cross-realm trust. Drop arcfour-hmac/des3 from permitted_enctypes —
  RC4 hard-removed in JDK 21 and service account now AES-only
  (msDS-SupportedEncryptionTypes=24). Fix hardcoded secret values in
  entrypoint.sh to use ${PROLE_KDC_MASTER_PASSWORD} variable references.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-27 20:06:36 -07:00
chrisfu
9523045add feat(oidc): fix knoe-auth routing for Kong strip_path=false; add Flyway baseline
Add @RequestMapping(/auth) to all OIDC and login controllers so paths
match the Kong route (strip_path: false keeps /auth in the forwarded path).

Changes:
- authority/web: add @RequestMapping(/auth) to LoginController,
  OidcAuthorizeController, OidcTokenController, OidcUserInfoController
- authority/resources: add V1__oidc_authorization_codes Flyway migration
- application-prole.yml: add OIDC clientId/clientSecret env bindings
- knoe-auth-deployment.yaml: add Flyway baseline env vars and
  KNOE_AUTH_OIDC_CLIENT_ID/SECRET from knoe-auth-oidc-client secret

Result: db.prole.org login flow redirects correctly through
https://api.prole.org/auth/login

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-26 02:13:33 -07:00
chrisfu
cf33342500 feat(prole): bootstrap knoe-auth on k3s; tenant onboarding; cluster stabilisation
knoe-auth (prole.org k3s):
- Fix CNPG manifest drift: remove spec.backup.pluginConfiguration (CNPG 1.28 only),
  switch spec.certificates from serverTLSSecret to serverAltDNSNames
- Apply knoe-auth Round 1 schema + GRANTs manually (postInitSQL had never run on live cluster)
- Fix OIDC signing key generator: base64(DER) not base64(PEM) — OidcTokenService
  does Base64.decode() → PKCS8EncodedKeySpec which requires raw DER bytes
- Add OIDC controllers: authorize, token, userinfo, jwks, discovery
- Add prole Spring profile: cookieDomain, emailDomain, Kerberos config
- Add secret example templates: knoe-db-user, knoe-auth-oidc-signing, knoe-auth-google-prole
- Kong configmap: scope knoe-auth route to /auth prefix only

Tenant onboarding:
- Add etc/onboard_tenant.sh: provision/apply/rotate/status workflow backed by 1Password
  vaults; types: 'enterprise' (own Kerberos + domain) and 'tenant' (hosted, initContainer KDC)
- Provision 'Knoe Tenant - prole.org' vault; apply all 7 k8s secrets to knoe-system
- init_knoe_auth.sh: add explicit GRANT + ALTER DEFAULT PRIVILEGES for knoe role

Cluster stabilisation:
- gitea: roll back 14-day stuck rollout (RWO PVC + maxSurge=100% deadlock);
  patch deployment strategy to Recreate
- supabase: create supabase_admin role, _supabase db, _analytics schema, _realtime schema
  in CNPG — analytics and realtime had never connected since Helm install day 1
- knoe-db barman ObjectStore: add GCS-backed objectstore manifest + scheduled backup

Infrastructure:
- gandalf host_vars: k3s registry config
- pi host_vars: clean up stale entries
- knoe-db schemas: ekosystem.sql, ekosystem_objects.sql
- init_prole_app.sql: prole app DB initialisation

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-26 00:50:37 -07:00
chrisfu
5cfd24cd05 fix(auth): fix gitea krb5.conf KDC hostname: knoe-auth→auth
KDC Service is 'auth' (ports 88/749); 'knoe-auth' only exposes HTTP port 8080.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-08 12:18:22 -07:00
chrisfu
9fe954d30e feat(auth): KNOE.LOCAL realm; prole-kerberos-ad-dc ExternalName svc; Gitea reverse proxy auth
- knoe-auth-kerberos-configmap.yaml: PROLE.LOCAL → KNOE.LOCAL
- prole-kerberos-ad-dc-svc.yaml: ExternalName Service for PROLE.ORG Samba AD KDC
- kustomization.yaml: register gitea-spnego-proxy + prole-kerberos-ad-dc-svc
- gitea/deploy.sh build_helm_values(): add service.ENABLE_REVERSE_PROXY_AUTHENTICATION,
  ENABLE_REVERSE_PROXY_AUTO_REGISTRATION, REVERSE_PROXY_TRUSTED_PROXIES=10.42.0.0/16

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-08 12:13:46 -07:00
chrisfu
063667c0b7 fix(gitea-spnego): upgrade probes tcpSocket→httpGet; halve memory limits
Go proxy exposes /_healthz so use httpGet probes (tcpSocket was only a
connectivity check, not a health check). Also reduce memory request/limit
to 32Mi/64Mi — the Go binary is lean and in-cluster load is minimal.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-08 11:35:01 -07:00
chrisfu
2a2b16d0fa feat(gitea): add Kerberos SPNEGO proxy for git.prole.org SSO
Deploys an Apache + mod_auth_gssapi sidecar in the gitea namespace that
handles SPNEGO/Kerberos negotiation (HTTP/git.prole.org@PROLE.ORG) and
injects X-WEBAUTH-USER for Gitea reverse-proxy auto-login.

Identity chain:
  Browser → Traefik TLS → Kong → gitea-spnego-proxy:4000
    → GSSAPI negotiate → X-WEBAUTH-USER: <username>
    → gitea-http:3000 (reverse proxy auto-registration)

Changes:
- gitea-spnego-proxy.yaml: krb5.conf ConfigMap + Deployment + Service
- gitea/spnego-proxy/: Dockerfile and Apache vhost (build source)
- kong-configmap.yaml: route git.prole.org → gitea-spnego-proxy:4000

Gitea reverse proxy settings applied via helm upgrade:
  ENABLE_REVERSE_PROXY_AUTHENTICATION=true
  ENABLE_REVERSE_PROXY_AUTO_REGISTRATION=true
  REVERSE_PROXY_AUTHENTICATION_USER_HEADER=X-WEBAUTH-USER

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-08 02:53:05 -07:00
chrisfu
52667c1e5d feat(backup): add CNPG ScheduledBackup manifest; fix 6-field cron schedule
Adds the ScheduledBackup CR to source control. Fixes schedule from
5-field to 6-field format (0 0 3 * * *) — CNPG requires seconds as
first field. The broken schedule caused the reconcile loop that
generated 746 stale Backup CRs. Deleted 7 orphaned GitLab Garage
buckets (~8.5 GB) and resumed daily backups.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-08 01:35:49 -07:00
chrisfu
1892e137c1 feat(infra): replace GitLab with Gitea on git.prole.org
- Remove GitLab hostname/storage config from k3s.cfg; GitLab is gone.
- Update GITEA_HOSTNAME git-internal → git.prole.org
- Fix MONITORING_STORAGE_CLASS local-path → merlin-local-iscsi-prometheus
- Kong: route git.prole.org → gitea-http.gitea:3000
- Add svc-knoe-ingress.yaml as source-of-truth for the prole ingress (was
  missing; live object named svc-knoe-ingress, routes git.prole.org through
  Traefik → Kong → Gitea)
- DNS: add git.prole.org A record → 73.15.20.166 (public front-door)
- monitoring/kps-values-k3s.yaml: fix storage class for Prometheus/Alertmanager

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-07 01:25:32 -04:00
chrisfu
16cc163204 fix(k3s): disable auth_request in grafana-proxy; disable auth.proxy in Grafana
knoe-auth /auth/verify not yet implemented — auth_request returns 500,
blocking all Grafana access. Simplify nginx to plain passthrough; strip
inbound X-WEBAUTH-USER to prevent header forgery. Disable Grafana
auth.proxy until knoe-auth is ready. Google auth.google still active.
2026-05-06 18:16:00 -04:00
chrisfu
d3eb01826c fix(k3s): use knoe-auth service name in Kong config (not authority-prole-auth) 2026-05-06 18:03:14 -04:00
chrisfu
529c20153e feat(k3s): enable allowExternalNameServices in Traefik
Needed for the knoe-system/oauth2-proxy ExternalName service that routes
db.prole.org ingress to oauth2-proxy running in the supabase namespace.
2026-05-06 17:58:50 -04:00
chrisfu
8c0e3493c6 fix(k3s): add knoe-auth Service and fix nginx auth_request FQDN
authority-prole-auth deployment had no Service. Create knoe-auth ClusterIP
service selecting app=authority-prole-auth on port 8080 — consistent with
Kong's knoe-auth route URL. Update nginx configmap to use the same FQDN.
2026-05-06 17:51:08 -04:00
chrisfu
416318dada fix(k3s): add namespace: monitoring to grafana-proxy-configmap
Without this, kubectl apply without -n flag lands in default namespace
instead of monitoring, leaving the live configmap with the old
knoe-auth hostname.
2026-05-06 17:47:50 -04:00
chrisfu
08f0f0cb74 feat(k3s): add ExternalName service for oauth2-proxy in knoe-system
Traefik ingress (svc-knoe-ingress) routes db.prole.org to
service/oauth2-proxy:80 in knoe-system. oauth2-proxy itself runs in
supabase namespace, so this ExternalName alias bridges the gap.
2026-05-06 17:41:31 -04:00
chrisfu
73bfbc3a98 fix(k3s): fix Kong + grafana-proxy config for prole.org routing
- kong-configmap: sync to live content; fix grafana upstream to route through
  knoe-grafana-proxy (nginx auth proxy) instead of kps-grafana (stale service
  name from old helm release); add db.prole.org → oauth2-proxy route; fix
  knoe-auth FQDN to authority-prole-auth
- grafana-proxy-configmap: fix upstream to prometheus-grafana (helm release
  name is 'prometheus', not 'kps'); fix auth_request FQDN to
  authority-prole-auth.knoe-system; fix login redirect to api.prole.org
- grafana-proxy-{deployment,service}: add namespace: monitoring
2026-05-06 17:26:40 -04:00
chrisfu
885fa99a29 feat(auth): add Google Workspace OAuth (prole.org) for Grafana + Supabase Studio
- monitoring/kps-values-k3s.yaml: Grafana helm values for k3s homelab with
  dual auth (auth.proxy Kerberos + auth.google for prole.org Workspace)
- grafana-proxy-configmap.yaml: nginx passthrough for /grafana/login/google
  and /grafana/login to allow Google OAuth flow without knoe-auth redirect
- grafana-google-oidc-secret-prole.example.yaml: Secret template for Grafana
  Google OAuth client (svc.prole.org, Internal consent, prole.org Workspace)
- oauth2-proxy-google-oidc-secret-prole.example.yaml: Secret template for
  oauth2-proxy gating db.prole.org Studio
- oauth2-proxy-deployment-prole.yaml: k3s oauth2-proxy deployment for
  db.prole.org (prole.org domain, no BackendConfig)
- init_grafana_oauth_prole.sh: Bootstrap script for Grafana OAuth secret
- init_oauth2_proxy_prole.sh: Bootstrap script for Studio oauth2-proxy

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-06 16:22:49 -04:00
chrisfu
bb71cf68ea feat(auth): dual IdP for db.prole.org — Google OAuth + Kerberos SPNEGO
knoe-auth (Spring Boot OIDC provider):
- AuthProperties: add google.workspaceDomain and kerberos.servicePrincipal fields
- GoogleOAuthService: validate hd (hosted domain) claim; restrict to configured workspace
- LoginController: /login/google endpoint + SPNEGO negotiation entry point
- PrincipalNormalizer: map Kerberos principal (user@REALM) to knoe-auth user
- application.yml: add spring.security.kerberos and oauth2.client stubs (values
  injected at runtime from env / Kubernetes Secrets)
- knoe-auth-deployment.yaml: mount HTTP keytab Secret; add GOOGLE_PROLE_CLIENT_ID /
  GOOGLE_PROLE_CLIENT_SECRET env from oauth2-proxy-prole-secret
- knoe-auth-http-keytab-secret.example.yaml: example Secret for HTTP/<host> keytab

Kong (init_kong.sh):
- Add db.prole.org route in k3s mode block via oauth2-proxy upstream
- Mode-gate: only registered for k3s, excluded for k3d/k8s

Supabase / oauth2-proxy:
- New supabase/helm/oauth2-proxy Helm chart: gates Supabase Studio at db.prole.org
  with Google OAuth (email-domain=prole.org) + cookie settings for .prole.org domain
- values-k3s.yaml: k3s-specific overrides (upstream service, TLS, cookie domain)
- secret-example.yaml: placeholder for oauth2-proxy-prole-secret

Ansible:
- infrastructure/playbooks/kerberos_trust_setup.yml: automates samba-tool domain
  trust create on myrddin.prole.org for PROLE.LOCAL ↔ PROLE.ORG cross-realm trust

Test:
- GoogleLoginProleOrgTest: verifies hd=prole.org tokens are accepted; hd=other.com rejected

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-03 20:03:09 -07:00
chrisfu
82c1ff555f fix(k3d): rename prole-svc-kong → knoe-svc-kong, clean up stale resources
k3s manifests still used prole-svc-kong after the rebrand; init_kong.sh
was waiting on `rollout status deployment/knoe-svc-kong` which never
existed, hanging the Common Services milestone.

- Rename kong-deployment.yaml and kong-service.yaml: prole-svc-kong →
  knoe-svc-kong (labels, selector, configmap volume ref)
- Remove prole-svc-kong-configmap.yaml (static file replaced by
  init_kong.sh dynamic ConfigMap generation; hardcoded namespace and
  prole-branded routes were dead weight)
- init_kong.sh: add cleanup_legacy_prole_kong() called from action_update()
  to remove stale prole-era resources before deploying knoe-svc-kong
- init_kong.sh: add _is_host_claimed_by_other_ingress() helper; set
  include_gitea_host=0 for k3d mode (git is port-forward only, no public
  hostname) — keeps the safety-net pre-check for k3s
- actions.py: extend legacy namespace dedupe to sweep prole-svc-kong from
  both default and service_ns

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-03 00:24:02 -07:00
chrisfu
903f84f200 feat: ship Junie #3 (image rename) + Phase 2 OIDC GKE deploy + k3d chrisfu seed
Three independent lines of work landing in one commit because they were all
on disk together end-of-session and the cross-file edits (TODO, knoe-system,
junie/README) interleave cleanly:

1. **Queue #3 — image rename `knoe-authority` → `knoe-auth`** (Junie).
   Closes drift R6. The Maven artifact has been `knoe-auth.jar` since
   commit b355855; the deploy manifests now match.
     authority/Dockerfile.app                                         (NEW)
     deploy/gcp/gke/knoe-auth-deployment.yaml                         (3 image tags renamed)
     deploy/opentofu/k3s/manifests/knoe/knoe-auth-deployment.yaml     (2 image tags renamed)
     Makefile                                                         (docker-build-auth +
                                                                       docker-push-auth + REGISTRY/KNOE_AUTH_VERSION
                                                                       defaults)
     docs/plans/junie/03-image-rename-knoe-authority-to-knoe-auth.md  (brief, kept as design record)

2. **Phase 2 OIDC provider — GKE deploy** (Junie). Source landed via the
   merge that brought claude/crazy-bose-fec256 back; the k3d sandbox
   shipped earlier today (commit 93157b0). This commit completes the GKE
   path: Kong route `/auth` → knoe-auth, default values added, signing
   key wired via `knoe-auth-oidc` secret per the brief.
     supabase/helm/knoe-supabase/templates/kong/config.yaml           (Kong route /auth, gated
                                                                       on .Values.knoeAuth.enabled)
     supabase/helm/knoe-supabase/values.yaml                          (knoeAuth.enabled=false default;
                                                                       studioIngress block — also
                                                                       fixes the helm lint issue noted
                                                                       in earlier commit 03bb731)
     docs/plans/junie/phase2-oidc-gke-deploy.md                       (brief, kept as design record)

3. **k3d dev-user seed: `chrisfu` + `knoe_developer`** (Claude). Closes
   "I want auth as chrisfu@knoey.com to my local cnpg database" for the
   local dev loop. `etc/init_knoe_auth.sh --mode k3d` now runs
   `seed_dev_users_k3d()` after the schema bootstrap, creating:
     - `knoe_developer` group role with R/W on `knoe`+`public` (mirrors
       the GKE production layout from docs/db-access.md; was hand-rolled
       in production per the 2026-04-30 onboarding work, never baked into
       postInitTemplateSQL).
     - `chrisfu` LOGIN role with password `chrisfu-dev` (idempotent —
       resets on every `make k3d-knoe-up`, so the rebuild loop is
       deterministic). Granted into `knoe_developer`.
   From the host with port-forward up:
     PGPASSWORD=chrisfu-dev psql "postgresql://chrisfu@localhost:5432/knoe-db?sslmode=require"

Cross-cutting doc updates:
  docs/TODO.md            — Phase 2 GKE entry removed from "In progress"
                            (now "(none — all items shipped or paused)");
                            queue #3 + reality table R6 removed; pg_oauth
                            paused note flipped to "OIDC issuer now
                            deployed to GKE — pg_oauth can resume"; Done
                            section gets new entries for #3, Phase 2 GKE,
                            and chrisfu seed.
  docs/knoe-system.md     — Phase 2 GKE row → Shipped; pg_oauth → Ready;
                            knoe-auth-deployment.yaml note "knoe-auth:latest"
                            instead of "knoe-authority:latest"; "One-time
                            cluster setup" mentions the chrisfu seed.
  docs/local-dev-knoe-auth.md — "Verify psql connectivity" rewritten to
                            connect as chrisfu (was `knoe`); new "Reset /
                            rebuild loop" section explaining idempotency.
  docs/plans/junie/README.md — two new rows under Shipped: brief 03 and
                            phase2-oidc-gke-deploy.

Verification (the bits I ran locally):
  - bash -n etc/init_knoe_auth.sh                                     OK
  - The seed SQL is idempotent (DO blocks with EXISTS checks +
    ALTER ROLE on the password reset path)
  - The Kong /auth route is gated on .Values.knoeAuth.enabled — default
    false, no behavior change until someone flips it on per Junie's brief

Out of scope for this commit:
  - Actually flipping knoeAuth.enabled=true on the live GKE chart and
    rolling out — Junie's brief covers the runbook; needs the
    knoe-auth-oidc K8s secret populated from 1Password first
  - Rebuilding + pushing the new knoe-auth:latest image — `make
    docker-push-auth` is wired, just hasn't been run yet
  - Round 1.5 OpenBao transit-key encryption (still queued)

Co-authored-by: Junie <junie@jetbrains.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-02 13:07:40 -07:00
chrisfu
c3fae73de3 fix(cnpg,kong): wire cnpg-backup-sa, migrate PodMonitor, drop DASHBOARD consumer
Three Junie briefs landed in one commit:

#07 — Wire cnpg-backup-sa into CNPG cluster spec (drift R8)
  deploy/gcp/gke/knoe-db.yaml: add spec.serviceAccountName: cnpg-backup-sa
  (requires CNPG v1.29+, which is the live operator version).
  etc/init_cnpg_gke.sh: operator install URL now uses CNPG_OPERATOR_VERSION
  variable (default 1.29.0); new §11 patches knoe-db and
  knoe-db-barman-cloud RoleBindings to add cnpg-backup-sa as a subject
  if not already present — matching the 2026-04-29 live stabilization.

#13 — Migrate off deprecated enablePodMonitor + podMonitorRelabelings
  Both deprecated fields removed from deploy/gcp/gke/knoe-db.yaml
  spec.monitoring. New deploy/gcp/gke/knoe-db-podmonitor.yaml carries
  the PodMonitor with the cluster relabeling rule (cnpg.io/cluster pod
  label → cluster label; required for all 85 CNPG Grafana panels).
  Apply alongside knoe-db.yaml on next cluster patch.

#15 — Remove dead DASHBOARD consumer + basicauth_credentials
  supabase/helm/knoe-supabase:
  - wrapper.sh: drop DASHBOARD_USERNAME / DASHBOARD_PASSWORD envsubst lines
  - config.yaml: drop DASHBOARD consumer + basicauth_credentials block
  - kong/deployment.yaml: drop both DASHBOARD env-var secret refs
  - values.yaml: rename secret.dashboard → secret.openai (apiKey only;
    username/password dropped — no enforcer since commit 25f1b2e)
  - secrets/dashboard.yaml + _helpers.tpl: renamed to openai /
    supabase.secret.openai
  - studio/deployment.yaml: reads from secret.openai.apiKey
  - ci/example.yaml: updated to secret.openai.apiKey
  helm template confirms knoe-supabase-openai secret referenced; no
  DASHBOARD output.

docs/TODO.md: queue items #7, #13, #15 + drift row R8 archived to Done.

Co-authored-by: Junie <junie@jetbrains.com>
2026-05-02 03:08:34 -07:00
chrisfu
34a25dde63 chore(deploy): split knoe-db-backup-gcs manifest — SA only, drop legacy ScheduledBackup
deploy/gcp/gke/knoe-db-backup-gcs.yaml previously contained both the
cnpg-backup-sa ServiceAccount (wanted) and a ScheduledBackup using
method: barmanObjectStore (unwanted — legacy in-tree method being
removed in CNPG v1.30, and the live cluster already has a plugin-method
ScheduledBackup that would be duplicated on re-apply).

Verified no other consumers of knoe-db-daily in the repo (grep clean).
File is now SA-only, matching its filename's intent.

Part 1/2 of queue item #6 fix (docs/plans/junie/06-patch-garage-script-fixes.md).

Co-authored-by: Junie <junie@jetbrains.com>
2026-05-02 03:00:51 -07:00
chrisfu
fb7e8b7135 chore(k3s): rename prole-*.yaml manifests to knoe-* and align contents
Mirrors commit b355855 (auth-side rename). The kustomization at
deploy/opentofu/k3s/manifests/knoe/kustomization.yaml has been
referencing knoe-* names since the rebrand; kustomize was broken
on the k3s deploy mode until this lands.

Five git mv:
  prole-configmap.yaml           -> knoe-configmap.yaml
  prole-deployment.yaml          -> knoe-deployment.yaml
  prole-service.yaml             -> knoe-service.yaml
  prole-kdc-configmap.yaml       -> knoe-kdc-configmap.yaml
  prole-kdc-secrets.example.yaml -> knoe-kdc-secrets.example.yaml

Plus content sweep of knoe-kdc-configmap.yaml: replaced stale
PROLE.LOCAL/PROLE.ORG realm refs and PROLE_KDC_* env vars with
KNOE.DEV/KNOE_KDC_* to match the canonical GKE configmap.

Closes queue item #2 in docs/TODO.md (drift R5).

Co-authored-by: Junie <junie@jetbrains.com>
2026-05-02 02:58:03 -07:00
chrisfu
00f0ebec07 Merge claude/crazy-bose-fec256 into main
Bringing the long-running session-feature branch back into main in one
deliberate sweep. The branch carried the cluster work that's been live for
weeks (cross-cluster CNPG metrics, Grafana w/ Google OAuth, supabase
oauth2-proxy, cluster recovery, pg.0.knoe.dev + per-engineer onboarding,
GCS-backed CNPG backups via Workload Identity, the env-contamination
guard, the Junie brief queue, the cnpg-grafana CSRF + memory-request
fixes from today), while main accumulated Junie's parallel knoe-auth
Phase 2 OIDC work (full provider surface: discovery, authorize, token,
userinfo, JWKS, RS256 signing, code exchange, session services).

Key decision: the two branches did COMPETING rebrands off the same
starting point (5ba9b63, 2026-04-27):

  - claude branch (commit b355855, earlier): org.prole.authority.* →
                                              dev.knoe.auth.*
                                              (artifact renamed to
                                              knoe-auth.jar)
  - main (commit 9daa94b, recent): org.prole.authority.* →
                                    dev.knoe.authority.*
                                    (kept "authority" artifact name)

dev.knoe.auth wins: cluster runs from this name, the Maven artifact is
already knoe-auth.jar, and the broader rename is the documented
namespace direction (per ~/.claude/projects/-Users-chrisfu-dev-knoe-db/
memory/MEMORY.md). All of main's recent Phase 2 OIDC content was ported
from authority/src/.../dev/knoe/authority/ into
authority/src/.../dev/knoe/auth/ with package declarations rewritten.

== File-level resolution summary ==

Textual conflicts (4):

  authority/pom.xml
    - Took our artifactId="auth"
    - Took our branch's removal of spring-security-kerberos-client
      (verified: Junie's Phase 2 OIDC code does not import it; the dep
      was already-dead config)

  docs/pipeline-phases.md
    - Took our branch's "Phase 1 not started" status. Main had a
      misplaced " Complete" with a knoe-auth-Phase-1 commit ref
      in the autobuild Phase 1 section — different domain.

  docs/plans/knoe-auth-round-1.md
    - Took our branch's dev.knoe.auth file table (vs main's
      dev.knoe.authority listing). Pure rename mismatch.

  supabase/helm/knoe-supabase/templates/kong/config.yaml
    - Took our branch's onboard route + plain dashboard wiring.
      Main had an oauth2proxy.enabled toggle that put oauth2-proxy as
      a Kong upstream — but the deployed architecture (commit 25f1b2e)
      has oauth2-proxy in FRONT of Kong, not behind. Main's wrapper
      reflected an architecture that was never deployed.
    - Took our branch's removal of basic-auth from dashboard route
      (queue #15 brief still tracks the matching values.yaml /
      kong/deployment.yaml cleanup).

Java tree reconciliation (44 file-pairs):

  20 dual-path source files + 2 dual-path tests
    Body-identical between main's authority/ and our branch's auth/
    after stripping package decls — main's commit 9daa94b was a pure
    rebrand. Took our branch's auth/ version for all 22.

  8 main-only source files (Phase 2 OIDC), ported into auth/:
    web/JwksController.java
    web/OidcAuthorizeController.java
    web/OidcDiscoveryController.java
    web/OidcTokenController.java
    web/OidcUserInfoController.java
    session/OidcCodeService.java
    session/OidcTokenService.java
    session/SessionService.java

  12 main-only test files, ported into auth/:
    HealthControllerTest.java
    enroll/EnrollValueTypesTest.java
    enroll/EnrollmentControllerTest.java
    enroll/TotpServiceTest.java
    kerberos/KadminClientTest.java
    kerberos/KerberosSpnegoResultTest.java
    web/LoginControllerTest.java
    admin/AdminControllerTest.java
    user/PrincipalNormalizerTest.java
    regression/IdentityRegressionTest.java
    session/OidcCodeServiceTest.java
    session/SessionServiceTest.java

  Port mechanics: read main:authority/...<file> via git show, then sed
  rewrite `package dev.knoe.authority` → `package dev.knoe.auth` and
  `import dev.knoe.authority` → `import dev.knoe.auth`. Body content
  unchanged.

  authority/src/main/java/dev/knoe/authority/ — DELETED (duplicate)
  authority/src/test/java/dev/knoe/authority/  — DELETED (duplicate)

== Verification ==

- grep -rln '<<<<<<<' across .java/.md/.yaml/.yml/.sh/.xml/.tpl: clean
- find authority/src -path '*/dev/knoe/authority*': empty (subtree gone)
- grep 'package dev.knoe.authority' across repo: clean
- bash -n install.sh deploy.sh etc/preflight_kubecontext.sh: clean
- git ls-files -u | wc -l: 0 unmerged paths
- helm lint supabase/helm/knoe-supabase: pre-existing failure on
  studioIngress.enabled undefined in values.yaml (introduced by Junie
  on main; unrelated to this merge — flagging as follow-up).

== Followups (carried into TODO ranked queue or noted here) ==

  - helm lint failure: studioIngress block in values.yaml is missing
    enable flag; templates/studio/{ingress,oauth2proxy-deployment,
    oauth2proxy-service}.yaml all reference studioIngress.enabled with
    no default. Pre-existing on main; not introduced by this merge.
  - The five Junie briefs filed on this branch are now reachable from
    main at docs/plans/junie/{02,06,07,13,15}-*.md. Junie can pick them
    up in any order.
  - knoe-auth Phase 2 OIDC source (now at dev.knoe.auth.*) is not yet
    deployed to the cluster. Deployment is its own task.
  - The branch claude/crazy-bose-fec256 stays in place (worktree at
    .claude/worktrees/crazy-bose-fec256 may have ongoing context for
    Claude Code sessions). Safe to delete once next session starts
    cleanly from main.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-01 16:39:10 -07:00
chrisfu
3728889e25 Phase 1: OIDC provider integration and GKE auth deployment
- Implement Google OIDC support in Authority module via GoogleOAuthService

- Update AuthProperties and application.yml with OIDC configuration

- Add oidc-setup.md documentation for GKE/Google Cloud setup

- Update etc/init_knoe_auth.sh to handle OIDC secrets and path-B configuration

- Configure knoe-auth-deployment.yaml and gke.cfg for production auth

Co-authored-by: Junie <junie@jetbrains.com>
2026-04-28 12:22:45 -07:00
chrisfu
d17270bbe2 feat(oauth): add Google OIDC/OAuth2 Proxy support for Studio and Grafana. Add Google OIDC configuration for Grafana in gke.cfg, introduce oauth2-proxy templates and configurations for Supabase Studio, update Helm values/templates for authenticated Studio access, and add ingress/service manifests for Postgres TCP and Prole services.
Co-authored-by: Junie <junie@jetbrains.com>
2026-04-28 11:27:34 -07:00
chrisfu
eb8c952523 chore(deploy): cluster config and k3s manifest updates
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-27 14:47:27 -07:00
chrisfu
40ea30e4c3 feat(auth): init scripts and k3s/k8s auth manifests for knoe-auth
init_knoe_auth.sh: provisions KDC secrets via 1Password, applies GKE manifests
init_knoe_users.sh: creates Kerberos principals for initial contributors
kerberos-configmap.yaml: krb5.conf for OpenBao Kerberos auth (KNOE.DEV realm)
prole-auth-deployment.yaml: k3s auth + kdc sidecar deployment for homelab
prole-kdc-configmap.yaml: k3s KDC config for homelab

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-27 14:47:27 -07:00
chrisfu
1417bc51f0 feat(auth): land Round 1 — invite-OTP enrollment, kadmin client, GKE manifests
Java implementation of the knoe-auth identity backbone (Round 1):

  authority/.../admin/         user admin REST endpoints (list, deactivate,
                               reset-totp, role management)
  authority/.../enroll/        invite-OTP web enrollment flow — accepts invite
                               token, creates Kerberos principal via kadmin,
                               seeds TOTP secret, returns keytab
  authority/.../provisioning/  contributor provisioning service — orchestrates
                               invite generation, principal lifecycle, role grants
  authority/.../kerberos/
    KadminClient.java          typed wrapper around the kadmin(1) subprocess;
                               create/delete/get-keytab/change-password

GKE manifests:

  deploy/gcp/gke/knoe-auth-deployment.yaml  two-container Pod (knoe-auth HTTP
                                             + kdc sidecar) with keytab-bootstrap
                                             initContainer
  deploy/gcp/gke/knoe-kdc-configmap.yaml    krb5.conf, kdc.conf, kadm5.acl and
                                             entrypoint for the embedded KDC
  deploy/gcp/gke/knoe-kdc-secrets.yaml      placeholder template — real values
                                             created by init_knoe_auth.sh via 1Password

See docs/plans/knoe-auth-round-1.md for full architectural narrative.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-27 14:47:27 -07:00
chrisfu
55b6a6aff3 Complete rebranding from prole to knoe and fix macOS application identity. Bulk renamed 'prole' to 'knoe' across code, scripts, and manifests. Updated Makefile with 'knoe', 'build', and 'start' targets. Implemented macOS Application Bundle (.app) support for correct identity. Fixed macOS 'Python' process name to 'Knoe.DB Installer' via Objective-C bridge. Standardized application name to 'Knoe.DB Installer' across all interfaces.
Co-authored-by: Junie <junie@jetbrains.com>
2026-04-22 15:08:35 -07:00
chrisfu
15df3ecc08 chore: purge ssl/, tfstate, ad-hoc artifacts; harden .gitignore
Follow-up to the ansible excision: remove other artifacts that should
never have been tracked, and extend .gitignore so they do not come back.

Removed:
- ssl/prole/*.key, *.crt, *.srl -- prole.org-era private keys, registry
  and DB CA material. References in knoe/core/env.py (auto-detect path
  at :1488) are best-effort inside try/except, so deletion is safe.
- ssl/prole.ssl.tar -- tarball of the same material.
- deploy/opentofu/k3s/terraform.tfstate -- 114KB committed OpenTofu
  state snapshot with live cluster UIDs. tfstate is never supposed to
  live in git.
- docs/deploy_gke.log, docs/deploy_gke_done.log,
  docs/compressed-history.log -- deploy-time log captures.
- end_time.txt, start_time.txt, start_time_final.txt -- ansible wall-
  clock timing breadcrumbs.
- network_description.txt, network_prompt.txt -- local scratch prompts.
- sidekiq_logs.txt -- transient log capture.
- test_resolve.sh -- ad-hoc debug script.

.gitignore additions:
- *.tfstate, *.tfstate.backup
- /ssl/
- .ai/
- /end_time.txt, /start_time*.txt, /sidekiq_logs.txt,
  /network_description.txt, /network_prompt.txt, /test_resolve.sh,
  /docs/deploy_*.log, /docs/compressed-history.log

Note: the purged SSL keys remain in git history until the deferred
history-scrub pass (task #14). The repo will be created as Private in
git.knoe.dev so this is acceptable for now; it MUST be scrubbed before
any broader visibility change.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-20 19:01:03 -07:00
chrisfu
e44ff8e476 fix(gitlab): unblock standalone reconcile + OIDC + object storage
Enables iterating on GitLab reconciles without a full deploy.sh run and
unblocks the end-to-end login flow:

- etc/init_gitlab.sh
  * Pre-scan $@ for --config before sourcing prole_cfg.sh so
    PROLE_DEPLOY_CFG is set for standalone invocations; previously
    prole_cfg.sh auto-picked conf/k3d.cfg and leaked KUBECONTEXT=dev,
    failing with `error: context "dev" does not exist`.
  * _init_gitlab_resolve_secretref: bash mirror of
    knoe/core/actions.py:_resolve_secretref_value so
    secretref://google-oidc-client-* resolves from etc/secrets/ files
    when run outside deploy.sh Python env-injection.
  * Garage bucket array: drop -storage suffix from uploads/artifacts/
    lfs/packages/dependency-proxy. Chart defaults for these have no
    suffix, and the object_store block in the CR does not override
    per-object bucket names; the prior mismatch caused first-login 500s
    (NoSuchBucket on avatar PUT).
- deploy/gcp/gke/gitlab-google-oidc-secret.example.yaml
  * Add discovery: true so omniauth-openid_connect fetches Google
    .well-known/openid-configuration; fixes "Could not authenticate
    from OpenIDConnect: No host info" on the callback.
- conf/gke.cfg
  * GITLAB_WEBSERVICE_LIMITS_MEMORY 1800M -> 3Gi, REQUESTS_MEMORY
    900M -> 2Gi, REQUESTS_CPU 200m -> 500m. Live pod was sitting at
    1706Mi/1800M (99%) in OOMKilled loop.
- conf/port-mapping.cfg: add supabase + gitea forwards, fix postgres
  namespace knoe-db -> knoe-db-0 for split-cluster CNPG layout.

Adds read-only diagnostics used to chase the above:
- etc/diag_gitlab_boot.sh, diag_gitlab_webservice_oom.sh, diag_gke_storage.sh
- etc/ensure_default_storage_class.sh + k8s/prole/storageclass-gcp-standard-hdd.yaml
  (preflight + HDD-default SC manifest for SSD-quota-constrained GKE projects).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-20 18:51:41 -07:00
chrisfu
0a799ca191 chore: refine post-remediation checks in init_gitlab.sh for GitLab workload convergence
- Introduced `gitlab_post_remediation_success` flag to streamline corrective action logging.
- Adjusted logic to skip unnecessary workload convergence gates when remediation is successful.
2026-04-18 15:48:36 -07:00
chrisfu
a2de7213ff chore: disable PodDisruptionBudget for Knoe-DB across all deployments 2026-04-15 21:20:43 -07:00
chrisfu
65369811f7 Harden ingress targeting and normalize HOME paths
- enforce app-cluster-only rendering for public ingress hosts with DB-cluster guardrails\n- fix Supabase/GitLab/authority ingress host ownership and ingress-class safety checks\n- normalize persisted home-directory paths to /Users/chrisfu and update gke config defaults\n- add/adjust regression tests for ingress placement/hostname and cfg path normalization

Co-authored-by: Junie <junie@jetbrains.com>
2026-04-12 12:32:09 -07:00
chrisfu
b1755f4e7b Stabilize prod deployment flow and harden Garage/CNPG ops
Switch production config to k8s/GKE contexts and align service naming. Add immutable StatefulSet update fallback for Garage across k3d/k3s/k8s. Harden CNPG deploy and backup bootstrap paths, and update installer coverage for CNPG webhook and Garage common ops.

Co-authored-by: Junie <junie@jetbrains.com>
2026-04-11 20:19:36 -07:00
chrisfu
8e5cf9c9bc feat: config-driven CNPG cluster registry
Add [CNPG Clusters] section to prole.cfg as the authoritative source for
CNPG cluster definitions. The deploy pipeline reads this registry instead
of computing version-appended namespace names at runtime.

- conf/service/prole.cfg: add [CNPG Clusters] with knoe-db as identity
  cluster; fix DATABASE_NAMESPACE=knoe-db (was knoe-db-18-140); add
  KNOE_IMAGE_REGISTRY for the k3s in-cluster registry hostname
- deploy/opentofu/k3s/manifests/prole/knoe-db.yaml: replace hardcoded
  registry.test-system image with templated KNOE_IMAGE_REGISTRY
- knoe/core/actions.py: add _cnpg_cluster_registry(), _cnpg_identity_cluster();
  fix _initial_db_namespace() to use declared namespace without version suffix
- knoe/core/milestones.py: iterate over all declared clusters in Init and
  Deploy milestones; per-cluster CNPG_IMAGE_NAME support
- knoe/core/ops/cloudnative_pg.py: resolve KNOE_IMAGE_REGISTRY and
  CNPG_IMAGE_NAME overrides in _apply_manifest()

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-11 10:56:12 -07:00
chrisfu
5460c58eb9 fix: remove Kerberos fields from k3s CNPG manifest
Drop spec.additionalVolumes, krb_server_keyfile, and hostgssenc pg_hba
rules from the k3s Cluster manifest. The CNPG CRD on k3s rejects
additionalVolumes as an unknown field, the keytab secret does not exist,
and hostgssenc rules would prevent PostgreSQL from starting without a
mounted keytab. Kerberos is disabled in conf/service/prole.cfg and
will be wired up once k3s KDC and keytab provisioning are in place.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-11 09:54:51 -07:00
chrisfu
a14fa654e6 chore: checkpoint deployment and identity bootstrap updates
- switch default git provider to GitLab and add cluster kubecontext settings in prole config

- add knoe user/user_role schema grants in GKE and k3s database manifests

- extend init_knoe_users bootstrap to create schema tables and provision default users

- make deploy pipeline honor configured cluster env and gate GKE-only defaults

- update GitLab OIDC default frontdoor host and refresh generated manifest/db metadata

Co-authored-by: Junie <junie@jetbrains.com>
2026-04-10 20:55:30 -07:00
chrisfu
c94c62e1bb Harden prod deploy namespace/context routing and vault password handling
- persist and load DB master password via Ansible Vault bootstrap flow

- enforce knoe-system service namespace and explicit app/db kubecontext targeting

- improve OpenBao/CNPG deploy reliability and logging; add retries/readiness diagnostics

- tighten reset/delete cluster behavior and expand installer/deploy pipeline test coverage

Co-authored-by: Junie <junie@jetbrains.com>
2026-04-10 00:43:07 -07:00
chrisfu
5cd43c873c feat(gke): finalize dual-cluster app/db topology
- separate default app and db cluster contexts (knoe-dev-0 and knoe-cnpg-0) across actions, ops flows, and installer config\n- update CNPG placement and GKE manifests to keep PostgreSQL on the db cluster while app services stay on the app cluster\n- add GCS-backed Barman ObjectStore manifest and cross-cluster Garage patch workflow\n- refresh UI cluster/database/service screens and monitoring wiring for dual-cluster operation\n- add reset/patch scripts, db context selection test coverage, and architecture/network documentation updates

Co-authored-by: Junie <junie@jetbrains.com>
2026-04-08 23:12:51 -07:00
chrisfu
64e1cef16c checkpoint: stabilize k8s cluster setup and shelve cluster-storage UI
- make cluster-storage milestone opt-in and remove installer cluster-storage step from UI navigation\n- add cluster storage browser and GKE cluster ops helpers with CLI coverage\n- update k8s/CNPG config and install flow files for corrected cluster setup\n- add/refresh tests for storage browser, GKE ops, prod config, and service-layer navigation

Co-authored-by: Junie <junie@jetbrains.com>
2026-04-07 21:11:25 -07:00
chrisfu
df6de9138e checkpoint: summarize CNPG init and backup hardening work
- harden CNPG backup ObjectStore readiness fallback in statusless clusters by validating controller availability and expected spec

- improve backup/plugin reconciliation paths and related test coverage for barman-cloud flows

- include ongoing GKE/CNPG deployment and configuration updates from this checkpoint

Co-authored-by: Junie <junie@jetbrains.com>
2026-04-06 05:51:26 -07:00
chrisfu
c165ee1a87 fix(cnpg): stabilize CNPG/Barman Cloud init on GKE (knoe-dev-0 / knoe-db-0)
- cloudnative_pg.py: add _wait_for_webhook_ca_bundle() to poll ValidatingWebhookConfiguration
  until cainjector has injected the CA bundle; fall back to _try_inject_ca_bundle_manually()
  on timeout (reads cert-manager-webhook-ca secret, patches webhook directly)
- cloudnative_pg.py: add _wait_for_barman_tls_secrets() to wait for barman-cloud-{client,server}-tls
  before declaring Barman Cloud ready; surfaces warning instead of silent FailedMount
- deploy/gcp/gke/namespace.yaml: ecosystem-0 -> knoe-db-0 (ekosystem = k8s cluster, not namespace)
- deploy/gcp/gke/knoe-db.yaml: fix namespace ecosystem-0 -> knoe-db-0; set
  podAntiAffinityType: required (one CNPG pod per node); add Spot node toleration so
  GKE autoscaler simulation succeeds and scale-up is unblocked; add resource
  requests/limits (500m/512Mi -> 2/2Gi); set googleCredentials.gkeEnvironment: true
  (use Workload Identity - no explicit credentials secret needed)
- deploy/gcp/gke/knoe-db-backup-gcs.yaml: fix namespaces ecosystem-0 -> knoe-db-0
- deploy/gcp/gke/workload-identity.yaml: update comments ecosystem-0 -> knoe-db-0
- tests/installer/test_cnpg_barman_webhook.py: new - 13 tests covering webhook CA bundle
  polling, manual CA injection, Barman TLS secret wait, and install_barman_plugin retry logic

Co-authored-by: Junie <junie@jetbrains.com>
2026-04-05 02:04:11 -07:00
chrisfu
3312c39b1f feat: GCP/GKE CNPG hardening, Artifact Registry traffic light, and knoe-system namespace fixes
UI screens
- database.py: fix mode detection to use env_key priority (prod→k8s, service→k3s) so stale DEPLOYMENT_MODE never overrides the user's chosen environment
- database.py: Registry status reads ARTIFACT_REGISTRY_AVAILABLE persisted by cluster screen; uses SERVICE_NAMESPACE for Artifact Registry repo name
- cluster.py: add Artifact Registry traffic light (amber→green/red) to prod section; _check_artifact_registry_async persists ARTIFACT_REGISTRY_AVAILABLE into Global cfg
- cluster.py: re-trigger Artifact Registry check after GKE cluster selection so the light re-evaluates once region is available from KUBECONTEXT
- cluster_nodes.py: fix TclError on Python 3.14 — pady=(2,0) tuple → pady=2 scalar
- __init__.py: seed knoe-system namespace when saved value is "default", not only when empty
- services.py: replace hardcoded "Prole DB" log string with dynamic cnpg_cluster name

Core ops
- cloudnative_pg.py: replace one-shot Barman plugin retry with 6-attempt loop; first cert-manager/x509 failure triggers rollout restart + 30 s CA propagation wait; subsequent failures back off up to 60 s per attempt
- cloudnative_pg.py: TLS CA CN now uses cluster_name instead of hardcoded "Prole CNPG CA"
- registry.py, garage_store.py: refactored into per-mode modules (k3d/k3s/k8s registry and garage store, shared _garage_common)

Deploy / config
- deploy/gcp/gke/knoe-db.yaml: GKE-specific CNPG cluster manifest (rw/ro/r on separate nodes with premium-rwo storage)
- etc/init_common_services.sh, modes/k8s/knoe-db/.version: updated for current deploy
- kong-deployment.yaml: updated manifest

Tests
- test_cluster_nodes_render_smoke.py: add pack/grid, winfo_children, winfo_reqheight, update_idletasks, grid_slaves to dummy widgets; monkeypatch tk.Label so CNPG placement render completes without a real Tkinter root

Co-authored-by: Junie <junie@jetbrains.com>
2026-04-04 19:36:08 -07:00
chrisfu
4a8d9cc90d feat: full GKE/prod deployment pipeline from UI to Artifact Registry
## GCP / Cluster Environment Screen
- Auto-populate Cloud tab from conf/prod/gcp.cfg on screen open (org_id,
  billing_account, billing_project, project_id)
- gcloud auth validity checked on screen startup; friendly modal dialog
  streams gcloud auth login output live so user never leaves the app
- Live GKE cluster browser: fetches clusters via gcloud container clusters
  list, displays with checkmark selector, auto-selects saved cluster
- Selecting a cluster runs get-credentials, sets KUBECONFIG/KUBECONTEXT,
  and syncs the region dropdown to the selected cluster's location
- Region dropdown populated live from gcloud compute regions list with
  checkmark on currently selected region; graceful fallback when offline
- New 'GCP Storage' tab with workload->StorageClass mapping (CNPG->premium-rwo,
  Redis/Monitoring->standard-rwo, Garage->garage-hdd) and Fetch from Cluster
- Provider readonly field styled correctly (no solid-black on macOS)
- Stale prole.cfg/conf/prole.cfg symlinks removed; all config I/O now
  resolves env-specific paths via prole_conf.entrypoint_path()

## GKE Autopilot Compatibility (Common Services)
- Synology iSCSI StorageClass and static PVs guarded behind PROLE_MODE!=k8s
  in init_openbao.sh (GKE Autopilot forbids hostPath/iSCSI volumes)
- In-cluster Docker registry (hostPath) skipped in k8s mode; GCP Artifact
  Registry used instead
- Kong renamed knoe-svc-kong in k8s mode; all health-check kubectl calls in
  init_common_services.sh and status_common_services.sh updated accordingly
- DNS endpoints switched from *.prole.org to *.knoe.dev in k8s mode
  (api.knoe.dev, git.knoe.dev, svc.knoe.dev); ingress uses gce class
- New GKE-clean Kong manifests under deploy/opentofu/k8s/manifests/prole/:
  no k3s node affinity, explicit Autopilot resource requests/limits

## Garage S3 Store (GKE)
- New garage-statefulset-gcp.yaml targeting garage-hdd StorageClass
  (pd-standard, avoids SSD_TOTAL_GB quota exhaustion in us-west3)
- New storageclass-gcp-hdd.yaml (pd-standard, Retain, WaitForFirstConsumer)
- GCP StorageClass manifests skipped on re-runs (Autopilot built-ins are
  immutable; skip-if-exists guard added)
- PVC deletion guard extended to cover any storageClass (not just synology)
  so stale claims are cleaned before StatefulSet recreation

## Topology (GKE Autopilot)
- DaemonSet collector skipped in prod mode (forbidden in kube-system by
  GKE Warden); Kubernetes-only node facts path used instead
- All ready GKE nodes assumed cnpg-eligible and monitoring-eligible without
  taint/synology-mount checks (skip_collector + assume_nodes_eligible flags)

## KUBECONFIG / kubectl (k8s mode)
- actions.py: new elif mode==k8s branch sets KUBECONFIG=~/.kube/config
  and injects KUBECONTEXT from prole_cfg_data into script env
- _build_kubectl_cmd falls back to Global.KUBECONTEXT when
  init_cluster.selected_kubectx is empty
- _activate_selected_gke_cluster persists KUBECONFIG/KUBECONTEXT to
  prole_cfg_data and saves prole.cfg immediately after get-credentials

## Database Build Screen (GKE)
- Registry display shows correct Artifact Registry URL
  (<region>-docker.pkg.dev/<project>/<namespace>/knoe-db) in green
- Build+push: gcloud auth configure-docker, auto-creates AR repository
  named after SERVICE_NAMESPACE (e.g. knoe-system) if missing, then
  docker tag + push; falls back to gcr.io if region unavailable
- GCP config loaded from conf/prod/gcp.cfg on every screen entry;
  keys normalised to lowercase so project_id lookup is always consistent

## Config / Namespace persistence
- prole_conf.py activate_environment: symlink creation removed; sets
  CLUSTER_ENV env-var so all subsequent calls resolve correct env directory
- knoe/ui/screens/__init__.py: startup config load uses entrypoint_path()
  instead of hardcoded conf/prole.cfg; seeds SERVICE_NAMESPACE=knoe-system
  for managed envs so Common Services never defaults to 'default'
- cfg.py _save_prole_cfg: saves to env-specific path via entrypoint_path()
- etc/prole_cfg.sh: removed all ln -snf symlink creation

Co-authored-by: Junie <junie@jetbrains.com>
2026-04-04 12:38:16 -07:00
chrisfu
eb9430df9d fix(gitlab,infra): ARM64 RPi service cluster – GitLab deploy in gitlab ns on gandalf
Namespace & routing
- milestones.py: GitOpsMilestone now resolves namespace from
  gitops.gitlab_namespace (new) → Global.GITLAB_NAMESPACE → 'gitlab'
  hardcoded; never falls through to gitops.namespace (was 'gitea')
- prole.cfg: add gitops.gitlab_namespace=gitlab + GITLAB_NAMESPACE=gitlab
- init_gitlab.sh: NAMESPACE defaults to gitlab, NODE_SELECTOR blanked so
  only gitaly+minio are node-pinned; GITOPS_NAMESPACE fallback removed

GitLab on ARM64 RPi (16 KB kernel pages)
- init_gitlab.sh: DaemonSet compiles jemalloc-5.3.0 with --with-lg-page=14
  (glibc/Ubuntu) on every node; LD_PRELOAD injected per Ruby component
- Minio: quay.io 2022 image (ARM64); configure init container replaced
  with ARM64 alpine that writes credential files; MINIO_ROOT_USER/PASSWORD
  injected directly into main container env via secretKeyRef
- Minio buckets auto-created post-deploy (registry, lfs, artifacts, etc.)
- webservice/sidekiq: replicaCount=1, reduced memory (1500M/800M),
  liveness probe initialDelaySeconds=3600 (Rails loads 25-40min on RPi)
- allowedHosts set as flat string list (chart 9.x default is list-of-maps
  which breaks URI initializer in 7_gitlab_http.rb)
- gitaly+minio always pinned to gandalf (local PV); other workloads spread

Storage
- Static PVs created for gitaly (50Gi) + minio (10Gi) on synology d005
- Synology dirs created before PVs; bucket creation idempotent

Redis (shared for GitLab KAS)
- init_redis.sh: persistence disabled (no dynamic provisioner); Redis used
  as pub/sub broker only

Infrastructure / pi.prole.org
- Removed pi.prole.org from [k3s_agents] – dedicated pihole node, OOM
- host_vars: k3s_enabled=false, k3s_state=absent (storage preserved)
- New playbook: infrastructure/playbooks/disable_pi_k3s.yml (drain + disable)
- monitoring.py: node-exporter DaemonSet excludes pi.prole.org
- init_monitoring.sh: pi.prole.org excluded from node-exporter affinity
- kong-deployment.yaml: affinity rule prevents scheduling on pi (pihole owns 80/443)

Co-authored-by: Junie <junie@jetbrains.com>
2026-04-02 20:15:11 -07:00