Commit Graph

23 Commits

Author SHA1 Message Date
chrisfu
d114801758 feat: wire ekosystem UUID system into CNPG manifests (Tasks 1, 3, 4)
Task 1 — ConfigMap + CNPG wiring:
- Add k8s/knoe/knoe-ekosystem-sql.yaml: ConfigMap embedding ekosystem.sql
  and ekosystem_objects.sql for CNPG postInitApplicationSQLRefs
- Add scripts/gen-ekosystem-configmap.py: generation script to keep the
  ConfigMap in sync with knoe-db/schema/ekosystem*.sql source files
- Add Makefile target: make k8s/knoe/knoe-ekosystem-sql.yaml
- Wire postInitApplicationSQLRefs into all three CNPG cluster manifests:
    k8s/knoe/knoe-db.yaml (k3s / prole-service-context production)
    deploy/gcp/gke/knoe-db.yaml (GKE)
    deploy/opentofu/k3s/manifests/knoe/knoe-db.yaml (OpenTofu k3s)
- Add knoe-ekosystem-sql.yaml to k8s/knoe/kustomization.yaml

Task 3 — Python counterpart utility:
- Add knoe/ekosystem.py: thread-safe EkosystemID generator matching the
  PostgreSQL bit layout [49:ts_ms|12:tenant|10:shard|11:seq], with
  decode() and can_access() helpers
- Add tests/test_ekosystem.py: 23 tests covering base36 encoding,
  round-trips, thread safety, can_access, and the spec round-trip assertion

Task 4 — knoe.user ekosystem_uuid column:
- Add ALTER TABLE knoe.user ADD COLUMN IF NOT EXISTS ekosystem_uuid text UNIQUE
  to postInitSQL in all three CNPG manifests

Task 2 (register prole tenant) requires a live DB connection — manual step.
Task 5 (LDAP/AD reconciler) is design-only per spec.

Co-authored-by: Junie <junie@jetbrains.com>
2026-05-30 00:26:48 -07:00
chrisfu
885fa99a29 feat(auth): add Google Workspace OAuth (prole.org) for Grafana + Supabase Studio
- monitoring/kps-values-k3s.yaml: Grafana helm values for k3s homelab with
  dual auth (auth.proxy Kerberos + auth.google for prole.org Workspace)
- grafana-proxy-configmap.yaml: nginx passthrough for /grafana/login/google
  and /grafana/login to allow Google OAuth flow without knoe-auth redirect
- grafana-google-oidc-secret-prole.example.yaml: Secret template for Grafana
  Google OAuth client (svc.prole.org, Internal consent, prole.org Workspace)
- oauth2-proxy-google-oidc-secret-prole.example.yaml: Secret template for
  oauth2-proxy gating db.prole.org Studio
- oauth2-proxy-deployment-prole.yaml: k3s oauth2-proxy deployment for
  db.prole.org (prole.org domain, no BackendConfig)
- init_grafana_oauth_prole.sh: Bootstrap script for Grafana OAuth secret
- init_oauth2_proxy_prole.sh: Bootstrap script for Studio oauth2-proxy

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-06 16:22:49 -04:00
chrisfu
903f84f200 feat: ship Junie #3 (image rename) + Phase 2 OIDC GKE deploy + k3d chrisfu seed
Three independent lines of work landing in one commit because they were all
on disk together end-of-session and the cross-file edits (TODO, knoe-system,
junie/README) interleave cleanly:

1. **Queue #3 — image rename `knoe-authority` → `knoe-auth`** (Junie).
   Closes drift R6. The Maven artifact has been `knoe-auth.jar` since
   commit b355855; the deploy manifests now match.
     authority/Dockerfile.app                                         (NEW)
     deploy/gcp/gke/knoe-auth-deployment.yaml                         (3 image tags renamed)
     deploy/opentofu/k3s/manifests/knoe/knoe-auth-deployment.yaml     (2 image tags renamed)
     Makefile                                                         (docker-build-auth +
                                                                       docker-push-auth + REGISTRY/KNOE_AUTH_VERSION
                                                                       defaults)
     docs/plans/junie/03-image-rename-knoe-authority-to-knoe-auth.md  (brief, kept as design record)

2. **Phase 2 OIDC provider — GKE deploy** (Junie). Source landed via the
   merge that brought claude/crazy-bose-fec256 back; the k3d sandbox
   shipped earlier today (commit 93157b0). This commit completes the GKE
   path: Kong route `/auth` → knoe-auth, default values added, signing
   key wired via `knoe-auth-oidc` secret per the brief.
     supabase/helm/knoe-supabase/templates/kong/config.yaml           (Kong route /auth, gated
                                                                       on .Values.knoeAuth.enabled)
     supabase/helm/knoe-supabase/values.yaml                          (knoeAuth.enabled=false default;
                                                                       studioIngress block — also
                                                                       fixes the helm lint issue noted
                                                                       in earlier commit 03bb731)
     docs/plans/junie/phase2-oidc-gke-deploy.md                       (brief, kept as design record)

3. **k3d dev-user seed: `chrisfu` + `knoe_developer`** (Claude). Closes
   "I want auth as chrisfu@knoey.com to my local cnpg database" for the
   local dev loop. `etc/init_knoe_auth.sh --mode k3d` now runs
   `seed_dev_users_k3d()` after the schema bootstrap, creating:
     - `knoe_developer` group role with R/W on `knoe`+`public` (mirrors
       the GKE production layout from docs/db-access.md; was hand-rolled
       in production per the 2026-04-30 onboarding work, never baked into
       postInitTemplateSQL).
     - `chrisfu` LOGIN role with password `chrisfu-dev` (idempotent —
       resets on every `make k3d-knoe-up`, so the rebuild loop is
       deterministic). Granted into `knoe_developer`.
   From the host with port-forward up:
     PGPASSWORD=chrisfu-dev psql "postgresql://chrisfu@localhost:5432/knoe-db?sslmode=require"

Cross-cutting doc updates:
  docs/TODO.md            — Phase 2 GKE entry removed from "In progress"
                            (now "(none — all items shipped or paused)");
                            queue #3 + reality table R6 removed; pg_oauth
                            paused note flipped to "OIDC issuer now
                            deployed to GKE — pg_oauth can resume"; Done
                            section gets new entries for #3, Phase 2 GKE,
                            and chrisfu seed.
  docs/knoe-system.md     — Phase 2 GKE row → Shipped; pg_oauth → Ready;
                            knoe-auth-deployment.yaml note "knoe-auth:latest"
                            instead of "knoe-authority:latest"; "One-time
                            cluster setup" mentions the chrisfu seed.
  docs/local-dev-knoe-auth.md — "Verify psql connectivity" rewritten to
                            connect as chrisfu (was `knoe`); new "Reset /
                            rebuild loop" section explaining idempotency.
  docs/plans/junie/README.md — two new rows under Shipped: brief 03 and
                            phase2-oidc-gke-deploy.

Verification (the bits I ran locally):
  - bash -n etc/init_knoe_auth.sh                                     OK
  - The seed SQL is idempotent (DO blocks with EXISTS checks +
    ALTER ROLE on the password reset path)
  - The Kong /auth route is gated on .Values.knoeAuth.enabled — default
    false, no behavior change until someone flips it on per Junie's brief

Out of scope for this commit:
  - Actually flipping knoeAuth.enabled=true on the live GKE chart and
    rolling out — Junie's brief covers the runbook; needs the
    knoe-auth-oidc K8s secret populated from 1Password first
  - Rebuilding + pushing the new knoe-auth:latest image — `make
    docker-push-auth` is wired, just hasn't been run yet
  - Round 1.5 OpenBao transit-key encryption (still queued)

Co-authored-by: Junie <junie@jetbrains.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-02 13:07:40 -07:00
chrisfu
c3fae73de3 fix(cnpg,kong): wire cnpg-backup-sa, migrate PodMonitor, drop DASHBOARD consumer
Three Junie briefs landed in one commit:

#07 — Wire cnpg-backup-sa into CNPG cluster spec (drift R8)
  deploy/gcp/gke/knoe-db.yaml: add spec.serviceAccountName: cnpg-backup-sa
  (requires CNPG v1.29+, which is the live operator version).
  etc/init_cnpg_gke.sh: operator install URL now uses CNPG_OPERATOR_VERSION
  variable (default 1.29.0); new §11 patches knoe-db and
  knoe-db-barman-cloud RoleBindings to add cnpg-backup-sa as a subject
  if not already present — matching the 2026-04-29 live stabilization.

#13 — Migrate off deprecated enablePodMonitor + podMonitorRelabelings
  Both deprecated fields removed from deploy/gcp/gke/knoe-db.yaml
  spec.monitoring. New deploy/gcp/gke/knoe-db-podmonitor.yaml carries
  the PodMonitor with the cluster relabeling rule (cnpg.io/cluster pod
  label → cluster label; required for all 85 CNPG Grafana panels).
  Apply alongside knoe-db.yaml on next cluster patch.

#15 — Remove dead DASHBOARD consumer + basicauth_credentials
  supabase/helm/knoe-supabase:
  - wrapper.sh: drop DASHBOARD_USERNAME / DASHBOARD_PASSWORD envsubst lines
  - config.yaml: drop DASHBOARD consumer + basicauth_credentials block
  - kong/deployment.yaml: drop both DASHBOARD env-var secret refs
  - values.yaml: rename secret.dashboard → secret.openai (apiKey only;
    username/password dropped — no enforcer since commit 25f1b2e)
  - secrets/dashboard.yaml + _helpers.tpl: renamed to openai /
    supabase.secret.openai
  - studio/deployment.yaml: reads from secret.openai.apiKey
  - ci/example.yaml: updated to secret.openai.apiKey
  helm template confirms knoe-supabase-openai secret referenced; no
  DASHBOARD output.

docs/TODO.md: queue items #7, #13, #15 + drift row R8 archived to Done.

Co-authored-by: Junie <junie@jetbrains.com>
2026-05-02 03:08:34 -07:00
chrisfu
34a25dde63 chore(deploy): split knoe-db-backup-gcs manifest — SA only, drop legacy ScheduledBackup
deploy/gcp/gke/knoe-db-backup-gcs.yaml previously contained both the
cnpg-backup-sa ServiceAccount (wanted) and a ScheduledBackup using
method: barmanObjectStore (unwanted — legacy in-tree method being
removed in CNPG v1.30, and the live cluster already has a plugin-method
ScheduledBackup that would be duplicated on re-apply).

Verified no other consumers of knoe-db-daily in the repo (grep clean).
File is now SA-only, matching its filename's intent.

Part 1/2 of queue item #6 fix (docs/plans/junie/06-patch-garage-script-fixes.md).

Co-authored-by: Junie <junie@jetbrains.com>
2026-05-02 03:00:51 -07:00
chrisfu
00f0ebec07 Merge claude/crazy-bose-fec256 into main
Bringing the long-running session-feature branch back into main in one
deliberate sweep. The branch carried the cluster work that's been live for
weeks (cross-cluster CNPG metrics, Grafana w/ Google OAuth, supabase
oauth2-proxy, cluster recovery, pg.0.knoe.dev + per-engineer onboarding,
GCS-backed CNPG backups via Workload Identity, the env-contamination
guard, the Junie brief queue, the cnpg-grafana CSRF + memory-request
fixes from today), while main accumulated Junie's parallel knoe-auth
Phase 2 OIDC work (full provider surface: discovery, authorize, token,
userinfo, JWKS, RS256 signing, code exchange, session services).

Key decision: the two branches did COMPETING rebrands off the same
starting point (5ba9b63, 2026-04-27):

  - claude branch (commit b355855, earlier): org.prole.authority.* →
                                              dev.knoe.auth.*
                                              (artifact renamed to
                                              knoe-auth.jar)
  - main (commit 9daa94b, recent): org.prole.authority.* →
                                    dev.knoe.authority.*
                                    (kept "authority" artifact name)

dev.knoe.auth wins: cluster runs from this name, the Maven artifact is
already knoe-auth.jar, and the broader rename is the documented
namespace direction (per ~/.claude/projects/-Users-chrisfu-dev-knoe-db/
memory/MEMORY.md). All of main's recent Phase 2 OIDC content was ported
from authority/src/.../dev/knoe/authority/ into
authority/src/.../dev/knoe/auth/ with package declarations rewritten.

== File-level resolution summary ==

Textual conflicts (4):

  authority/pom.xml
    - Took our artifactId="auth"
    - Took our branch's removal of spring-security-kerberos-client
      (verified: Junie's Phase 2 OIDC code does not import it; the dep
      was already-dead config)

  docs/pipeline-phases.md
    - Took our branch's "Phase 1 not started" status. Main had a
      misplaced " Complete" with a knoe-auth-Phase-1 commit ref
      in the autobuild Phase 1 section — different domain.

  docs/plans/knoe-auth-round-1.md
    - Took our branch's dev.knoe.auth file table (vs main's
      dev.knoe.authority listing). Pure rename mismatch.

  supabase/helm/knoe-supabase/templates/kong/config.yaml
    - Took our branch's onboard route + plain dashboard wiring.
      Main had an oauth2proxy.enabled toggle that put oauth2-proxy as
      a Kong upstream — but the deployed architecture (commit 25f1b2e)
      has oauth2-proxy in FRONT of Kong, not behind. Main's wrapper
      reflected an architecture that was never deployed.
    - Took our branch's removal of basic-auth from dashboard route
      (queue #15 brief still tracks the matching values.yaml /
      kong/deployment.yaml cleanup).

Java tree reconciliation (44 file-pairs):

  20 dual-path source files + 2 dual-path tests
    Body-identical between main's authority/ and our branch's auth/
    after stripping package decls — main's commit 9daa94b was a pure
    rebrand. Took our branch's auth/ version for all 22.

  8 main-only source files (Phase 2 OIDC), ported into auth/:
    web/JwksController.java
    web/OidcAuthorizeController.java
    web/OidcDiscoveryController.java
    web/OidcTokenController.java
    web/OidcUserInfoController.java
    session/OidcCodeService.java
    session/OidcTokenService.java
    session/SessionService.java

  12 main-only test files, ported into auth/:
    HealthControllerTest.java
    enroll/EnrollValueTypesTest.java
    enroll/EnrollmentControllerTest.java
    enroll/TotpServiceTest.java
    kerberos/KadminClientTest.java
    kerberos/KerberosSpnegoResultTest.java
    web/LoginControllerTest.java
    admin/AdminControllerTest.java
    user/PrincipalNormalizerTest.java
    regression/IdentityRegressionTest.java
    session/OidcCodeServiceTest.java
    session/SessionServiceTest.java

  Port mechanics: read main:authority/...<file> via git show, then sed
  rewrite `package dev.knoe.authority` → `package dev.knoe.auth` and
  `import dev.knoe.authority` → `import dev.knoe.auth`. Body content
  unchanged.

  authority/src/main/java/dev/knoe/authority/ — DELETED (duplicate)
  authority/src/test/java/dev/knoe/authority/  — DELETED (duplicate)

== Verification ==

- grep -rln '<<<<<<<' across .java/.md/.yaml/.yml/.sh/.xml/.tpl: clean
- find authority/src -path '*/dev/knoe/authority*': empty (subtree gone)
- grep 'package dev.knoe.authority' across repo: clean
- bash -n install.sh deploy.sh etc/preflight_kubecontext.sh: clean
- git ls-files -u | wc -l: 0 unmerged paths
- helm lint supabase/helm/knoe-supabase: pre-existing failure on
  studioIngress.enabled undefined in values.yaml (introduced by Junie
  on main; unrelated to this merge — flagging as follow-up).

== Followups (carried into TODO ranked queue or noted here) ==

  - helm lint failure: studioIngress block in values.yaml is missing
    enable flag; templates/studio/{ingress,oauth2proxy-deployment,
    oauth2proxy-service}.yaml all reference studioIngress.enabled with
    no default. Pre-existing on main; not introduced by this merge.
  - The five Junie briefs filed on this branch are now reachable from
    main at docs/plans/junie/{02,06,07,13,15}-*.md. Junie can pick them
    up in any order.
  - knoe-auth Phase 2 OIDC source (now at dev.knoe.auth.*) is not yet
    deployed to the cluster. Deployment is its own task.
  - The branch claude/crazy-bose-fec256 stays in place (worktree at
    .claude/worktrees/crazy-bose-fec256 may have ongoing context for
    Claude Code sessions). Safe to delete once next session starts
    cleanly from main.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-01 16:39:10 -07:00
chrisfu
3728889e25 Phase 1: OIDC provider integration and GKE auth deployment
- Implement Google OIDC support in Authority module via GoogleOAuthService

- Update AuthProperties and application.yml with OIDC configuration

- Add oidc-setup.md documentation for GKE/Google Cloud setup

- Update etc/init_knoe_auth.sh to handle OIDC secrets and path-B configuration

- Configure knoe-auth-deployment.yaml and gke.cfg for production auth

Co-authored-by: Junie <junie@jetbrains.com>
2026-04-28 12:22:45 -07:00
chrisfu
1417bc51f0 feat(auth): land Round 1 — invite-OTP enrollment, kadmin client, GKE manifests
Java implementation of the knoe-auth identity backbone (Round 1):

  authority/.../admin/         user admin REST endpoints (list, deactivate,
                               reset-totp, role management)
  authority/.../enroll/        invite-OTP web enrollment flow — accepts invite
                               token, creates Kerberos principal via kadmin,
                               seeds TOTP secret, returns keytab
  authority/.../provisioning/  contributor provisioning service — orchestrates
                               invite generation, principal lifecycle, role grants
  authority/.../kerberos/
    KadminClient.java          typed wrapper around the kadmin(1) subprocess;
                               create/delete/get-keytab/change-password

GKE manifests:

  deploy/gcp/gke/knoe-auth-deployment.yaml  two-container Pod (knoe-auth HTTP
                                             + kdc sidecar) with keytab-bootstrap
                                             initContainer
  deploy/gcp/gke/knoe-kdc-configmap.yaml    krb5.conf, kdc.conf, kadm5.acl and
                                             entrypoint for the embedded KDC
  deploy/gcp/gke/knoe-kdc-secrets.yaml      placeholder template — real values
                                             created by init_knoe_auth.sh via 1Password

See docs/plans/knoe-auth-round-1.md for full architectural narrative.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-27 14:47:27 -07:00
chrisfu
55b6a6aff3 Complete rebranding from prole to knoe and fix macOS application identity. Bulk renamed 'prole' to 'knoe' across code, scripts, and manifests. Updated Makefile with 'knoe', 'build', and 'start' targets. Implemented macOS Application Bundle (.app) support for correct identity. Fixed macOS 'Python' process name to 'Knoe.DB Installer' via Objective-C bridge. Standardized application name to 'Knoe.DB Installer' across all interfaces.
Co-authored-by: Junie <junie@jetbrains.com>
2026-04-22 15:08:35 -07:00
chrisfu
e44ff8e476 fix(gitlab): unblock standalone reconcile + OIDC + object storage
Enables iterating on GitLab reconciles without a full deploy.sh run and
unblocks the end-to-end login flow:

- etc/init_gitlab.sh
  * Pre-scan $@ for --config before sourcing prole_cfg.sh so
    PROLE_DEPLOY_CFG is set for standalone invocations; previously
    prole_cfg.sh auto-picked conf/k3d.cfg and leaked KUBECONTEXT=dev,
    failing with `error: context "dev" does not exist`.
  * _init_gitlab_resolve_secretref: bash mirror of
    knoe/core/actions.py:_resolve_secretref_value so
    secretref://google-oidc-client-* resolves from etc/secrets/ files
    when run outside deploy.sh Python env-injection.
  * Garage bucket array: drop -storage suffix from uploads/artifacts/
    lfs/packages/dependency-proxy. Chart defaults for these have no
    suffix, and the object_store block in the CR does not override
    per-object bucket names; the prior mismatch caused first-login 500s
    (NoSuchBucket on avatar PUT).
- deploy/gcp/gke/gitlab-google-oidc-secret.example.yaml
  * Add discovery: true so omniauth-openid_connect fetches Google
    .well-known/openid-configuration; fixes "Could not authenticate
    from OpenIDConnect: No host info" on the callback.
- conf/gke.cfg
  * GITLAB_WEBSERVICE_LIMITS_MEMORY 1800M -> 3Gi, REQUESTS_MEMORY
    900M -> 2Gi, REQUESTS_CPU 200m -> 500m. Live pod was sitting at
    1706Mi/1800M (99%) in OOMKilled loop.
- conf/port-mapping.cfg: add supabase + gitea forwards, fix postgres
  namespace knoe-db -> knoe-db-0 for split-cluster CNPG layout.

Adds read-only diagnostics used to chase the above:
- etc/diag_gitlab_boot.sh, diag_gitlab_webservice_oom.sh, diag_gke_storage.sh
- etc/ensure_default_storage_class.sh + k8s/prole/storageclass-gcp-standard-hdd.yaml
  (preflight + HDD-default SC manifest for SSD-quota-constrained GKE projects).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-20 18:51:41 -07:00
chrisfu
0a799ca191 chore: refine post-remediation checks in init_gitlab.sh for GitLab workload convergence
- Introduced `gitlab_post_remediation_success` flag to streamline corrective action logging.
- Adjusted logic to skip unnecessary workload convergence gates when remediation is successful.
2026-04-18 15:48:36 -07:00
chrisfu
a2de7213ff chore: disable PodDisruptionBudget for Knoe-DB across all deployments 2026-04-15 21:20:43 -07:00
chrisfu
a14fa654e6 chore: checkpoint deployment and identity bootstrap updates
- switch default git provider to GitLab and add cluster kubecontext settings in prole config

- add knoe user/user_role schema grants in GKE and k3s database manifests

- extend init_knoe_users bootstrap to create schema tables and provision default users

- make deploy pipeline honor configured cluster env and gate GKE-only defaults

- update GitLab OIDC default frontdoor host and refresh generated manifest/db metadata

Co-authored-by: Junie <junie@jetbrains.com>
2026-04-10 20:55:30 -07:00
chrisfu
c94c62e1bb Harden prod deploy namespace/context routing and vault password handling
- persist and load DB master password via Ansible Vault bootstrap flow

- enforce knoe-system service namespace and explicit app/db kubecontext targeting

- improve OpenBao/CNPG deploy reliability and logging; add retries/readiness diagnostics

- tighten reset/delete cluster behavior and expand installer/deploy pipeline test coverage

Co-authored-by: Junie <junie@jetbrains.com>
2026-04-10 00:43:07 -07:00
chrisfu
5cd43c873c feat(gke): finalize dual-cluster app/db topology
- separate default app and db cluster contexts (knoe-dev-0 and knoe-cnpg-0) across actions, ops flows, and installer config\n- update CNPG placement and GKE manifests to keep PostgreSQL on the db cluster while app services stay on the app cluster\n- add GCS-backed Barman ObjectStore manifest and cross-cluster Garage patch workflow\n- refresh UI cluster/database/service screens and monitoring wiring for dual-cluster operation\n- add reset/patch scripts, db context selection test coverage, and architecture/network documentation updates

Co-authored-by: Junie <junie@jetbrains.com>
2026-04-08 23:12:51 -07:00
chrisfu
64e1cef16c checkpoint: stabilize k8s cluster setup and shelve cluster-storage UI
- make cluster-storage milestone opt-in and remove installer cluster-storage step from UI navigation\n- add cluster storage browser and GKE cluster ops helpers with CLI coverage\n- update k8s/CNPG config and install flow files for corrected cluster setup\n- add/refresh tests for storage browser, GKE ops, prod config, and service-layer navigation

Co-authored-by: Junie <junie@jetbrains.com>
2026-04-07 21:11:25 -07:00
chrisfu
df6de9138e checkpoint: summarize CNPG init and backup hardening work
- harden CNPG backup ObjectStore readiness fallback in statusless clusters by validating controller availability and expected spec

- improve backup/plugin reconciliation paths and related test coverage for barman-cloud flows

- include ongoing GKE/CNPG deployment and configuration updates from this checkpoint

Co-authored-by: Junie <junie@jetbrains.com>
2026-04-06 05:51:26 -07:00
chrisfu
c165ee1a87 fix(cnpg): stabilize CNPG/Barman Cloud init on GKE (knoe-dev-0 / knoe-db-0)
- cloudnative_pg.py: add _wait_for_webhook_ca_bundle() to poll ValidatingWebhookConfiguration
  until cainjector has injected the CA bundle; fall back to _try_inject_ca_bundle_manually()
  on timeout (reads cert-manager-webhook-ca secret, patches webhook directly)
- cloudnative_pg.py: add _wait_for_barman_tls_secrets() to wait for barman-cloud-{client,server}-tls
  before declaring Barman Cloud ready; surfaces warning instead of silent FailedMount
- deploy/gcp/gke/namespace.yaml: ecosystem-0 -> knoe-db-0 (ekosystem = k8s cluster, not namespace)
- deploy/gcp/gke/knoe-db.yaml: fix namespace ecosystem-0 -> knoe-db-0; set
  podAntiAffinityType: required (one CNPG pod per node); add Spot node toleration so
  GKE autoscaler simulation succeeds and scale-up is unblocked; add resource
  requests/limits (500m/512Mi -> 2/2Gi); set googleCredentials.gkeEnvironment: true
  (use Workload Identity - no explicit credentials secret needed)
- deploy/gcp/gke/knoe-db-backup-gcs.yaml: fix namespaces ecosystem-0 -> knoe-db-0
- deploy/gcp/gke/workload-identity.yaml: update comments ecosystem-0 -> knoe-db-0
- tests/installer/test_cnpg_barman_webhook.py: new - 13 tests covering webhook CA bundle
  polling, manual CA injection, Barman TLS secret wait, and install_barman_plugin retry logic

Co-authored-by: Junie <junie@jetbrains.com>
2026-04-05 02:04:11 -07:00
chrisfu
3312c39b1f feat: GCP/GKE CNPG hardening, Artifact Registry traffic light, and knoe-system namespace fixes
UI screens
- database.py: fix mode detection to use env_key priority (prod→k8s, service→k3s) so stale DEPLOYMENT_MODE never overrides the user's chosen environment
- database.py: Registry status reads ARTIFACT_REGISTRY_AVAILABLE persisted by cluster screen; uses SERVICE_NAMESPACE for Artifact Registry repo name
- cluster.py: add Artifact Registry traffic light (amber→green/red) to prod section; _check_artifact_registry_async persists ARTIFACT_REGISTRY_AVAILABLE into Global cfg
- cluster.py: re-trigger Artifact Registry check after GKE cluster selection so the light re-evaluates once region is available from KUBECONTEXT
- cluster_nodes.py: fix TclError on Python 3.14 — pady=(2,0) tuple → pady=2 scalar
- __init__.py: seed knoe-system namespace when saved value is "default", not only when empty
- services.py: replace hardcoded "Prole DB" log string with dynamic cnpg_cluster name

Core ops
- cloudnative_pg.py: replace one-shot Barman plugin retry with 6-attempt loop; first cert-manager/x509 failure triggers rollout restart + 30 s CA propagation wait; subsequent failures back off up to 60 s per attempt
- cloudnative_pg.py: TLS CA CN now uses cluster_name instead of hardcoded "Prole CNPG CA"
- registry.py, garage_store.py: refactored into per-mode modules (k3d/k3s/k8s registry and garage store, shared _garage_common)

Deploy / config
- deploy/gcp/gke/knoe-db.yaml: GKE-specific CNPG cluster manifest (rw/ro/r on separate nodes with premium-rwo storage)
- etc/init_common_services.sh, modes/k8s/knoe-db/.version: updated for current deploy
- kong-deployment.yaml: updated manifest

Tests
- test_cluster_nodes_render_smoke.py: add pack/grid, winfo_children, winfo_reqheight, update_idletasks, grid_slaves to dummy widgets; monkeypatch tk.Label so CNPG placement render completes without a real Tkinter root

Co-authored-by: Junie <junie@jetbrains.com>
2026-04-04 19:36:08 -07:00
chrisfu
872d715864 feat: Kerberos identity management — PROLE.LOCAL realm, cross-realm trust, user provisioning
Kerberos is now the single source of truth for all knoe-system accounts.

In-cluster KDC (PROLE.LOCAL):
- Rename in-cluster KDC realm PROLE.ORG → PROLE.LOCAL to avoid collision
  with the myrddin.prole.org Samba AD DC which owns PROLE.ORG
- Add [capaths] cross-realm trust block: PROLE.ORG (Samba AD) ↔ PROLE.LOCAL
- Create admin@PROLE.LOCAL on KDC startup (master password) for UI login
- Create guest@PROLE.LOCAL on KDC startup (read-only, PROLE_KDC_GUEST_PASSWORD)
- Update prole-kdc-secrets.example.yaml with trust_shared_password and guest_password
- Update prole-auth-kerberos ConfigMap realm/servicePrincipal to PROLE.LOCAL
- Add PROLE_AUTH_ADMIN_PRINCIPALS and PROLE_KDC_GUEST_PASSWORD to prole-auth Deployment

Database (knoe-db — k3s and GKE):
- Add spec.managed.roles: admin (superuser), guest (login), developer (group)
- Enable GSS pg_hba rules for both PROLE.ORG and PROLE.LOCAL realms
- Add krb_server_keyfile parameter for postgres service principal keytab
- Mount knoe-db-pg-keytab Secret via spec.additionalVolumes
- Add demo schema in postInitSQL with guest read-only grants

prole-auth (Spring Boot):
- SessionUser: add List<String> groups field
- AuthProperties: add adminPrincipals list (env: PROLE_AUTH_ADMIN_PRINCIPALS)
- LoginController: stamp groups=["admin"] for configured admin principals at login
- SessionTokenService: carry groups through TokenPayload; @JsonIgnoreProperties
  for backward compat with existing 2-field session cookies
- VerifyController: emit X-Prole-Groups header on /auth/verify
- grafana-proxy nginx: strip inbound X-Prole-Groups; capture and forward from auth

Services:
- ArgoCD (k3s + GKE): add g, admin, role:admin to argocd-rbac-cm policy.csv

etc/init_knoe_users.sh (new):
- Orchestrates full user provisioning: KDC principals, postgres keytab export,
  CNPG managed.roles patch, demo schema SQL, ArgoCD RBAC patch,
  Gitea and GitLab admin promotion via API
- Actions: initialize | status | cleanup
- Idempotent; sources prole_cfg.sh; follows existing init_*.sh style

Cross-realm trust activation (myrddin side):
  samba-tool user add krbtgt_PROLELOCAL --random-password
  samba-tool user setpassword krbtgt_PROLELOCAL --newpassword=<trust_shared_password>
  samba-tool spn add "krbtgt/PROLE.LOCAL" krbtgt_PROLELOCAL -U administrator
  samba-tool user setexpiry krbtgt_PROLELOCAL --noexpiry

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-04-01 20:38:57 -07:00
chrisfu
620bdd25eb feat: GKE production deployment — CNPG + GCS backups + Google Workspace SSO
Add complete GKE deployment stack:
- deploy/gcp/gke/: namespace, CloudNativePG cluster (premium-rwo, 100Gi),
  GCS barman-cloud backup with Workload Identity SA, ArgoCD OIDC ConfigMap,
  GitLab OmniAuth secret template, knoe-auth Google OIDC secret template
- etc/init_cnpg_gke.sh: provision GCS buckets, GCP SA, WI binding, CNPG operator
- etc/init_argocd.sh: apply argocd-oidc-cm.yaml when PLATFORM_DOMAIN/FRONTDOOR_HOST set
- etc/init_gitlab.sh: inject OmniAuth openid_connect block when FRONTDOOR_HOST set
- prole-auth-deployment.yaml: add GOOGLE_OIDC_ENABLED env vars (all optional)
- supabase/helm/prole-supabase/values-gke.yaml: GoTrue Google OAuth overlay
- knoe/core/prod_config.py: add backupProvider, workloadIdentitySA, googleClientId/Secret/HostedDomain

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-01 10:41:55 -07:00
chrisfu
a069989315 Rename prole-db to knoe-db, add knoe-auth as cluster-internal KDC
Itemized changes:

1. knoe-auth: New cluster-internal KDC and SSO gateway service
   - Created etc/init_knoe_auth.sh based on init_kdc.sh with knoe-auth naming
   - Namespace defaults to SERVICE_NAMESPACE (knoe-system)
   - ConfigMap: knoe-auth-kdc-config, Secret: knoe-auth-secrets
   - Legacy cleanup removes old auth/dog/authority deployments

2. Orchestration: knoe-auth initializes before CloudNativePG
   - Updated prole.sh to insert init_knoe_auth.sh as step 2 (before CNPG)
   - Renumbered all subsequent initialization steps

3. Kong routing: Updated init_kong.sh to route to knoe-auth in SERVICE_NAMESPACE

4. Comment/reference updates for knoe-auth
   - Updated init_common_services.sh, init_service_layer.sh, init_kerberos.sh

5. prole-db renamed to knoe-db across the entire codebase
   - Renamed prole-db/ directory to knoe-db/
   - Renamed all prole-db Kubernetes manifests (deploy/opentofu, k8s/)
   - Renamed scripts: docker-root-knoe-db.sh, docker-run-knoe-db.sh, test-cnpg-knoe-db.sh
   - Renamed etc/init_prole-db-reset.sh to etc/init_knoe-db-reset.sh
   - Renamed etc/prole-db-passwwd.sh to etc/knoe-db-passwwd.sh
   - Renamed mock_val counterparts accordingly
   - Renamed tests/etc/test_init_prole-db-reset.sh to test_init_knoe-db-reset.sh
   - Renamed docs/prole-db-documentation-mcp-architecture.md to knoe-db variant
   - Renamed modes/k3d/prole-db/ to modes/k3d/knoe-db/
   - Renamed prole-db.iml to knoe-db.iml

6. Configuration updates
   - Updated conf/dev, conf/prod, conf/test, conf/service prole.cfg files
   - Updated conf/port-mapping.cfg
   - Updated etc/prole_cfg.sh and mock_val/prole_cfg.sh
   - Updated service/prole.cfg

7. Kubernetes manifests and deploy configuration
   - Updated deploy/opentofu/k3s ArgoCD application YAMLs
   - Updated kong-configmap.yaml and kustomization.yaml
   - Updated k3s/kong-config.yml and prole-resources.yaml
   - Updated prole-mssql-db deployment YAMLs
   - Updated supabase helm render and deploy scripts

8. Infrastructure and GCP Terraform
   - Updated deploy/gcp/terraform: folders, groups, IAM, service-projects

9. Python/installer code updates
   - Updated knoe/core: actions, build_context, controller, env, milestones
   - Updated knoe/milestone.py
   - Updated knoe/ui/screens: cfg, database, database_options, deploy, docker,
     navigation, security, services, validate
   - Updated knoe.spec, status.py

10. Shell script updates
    - Updated etc/: build_db, init_cloudnative_pg, init_cnpg_backup,
      init_db_manager, init_forgejo, init_gitlab, init_monitoring, init_openbao,
      init_port_forwards, init_postgrest, init_supabase_ports, status
    - Updated mock_val/ counterparts for all above scripts
    - Updated prole-net/init-prole-dns.sh
    - Updated bin/prole-kpf.sh, gitea/deploy.sh, supabase/deploy.sh

11. Test updates
    - Updated tests/etc/: test_init_cloudnative_pg*, test_init_cnpg_backup*,
      test_init_kdc*, test_init_kerberos*, test_init_kong*, test_prole_cfg*
    - Updated tests/installer/: test_actions_helpers, test_cfg_save_kubecontext,
      test_controller, test_core_classes, test_milestones, test_milestones_extended,
      test_namespace_propagation
    - Updated tests/: test_database_options, test_navigation,
      test_render_supabase_hostname, test_docker_build_fix,
      test_all_prole_home_fixes, silent_install_test, final_test

12. Documentation updates
    - Updated docs/: DOCKER-BUILD-FIX, PROLE-CFG-SECRETS, PROLE-HOME-DIRECTORY,
      build-system, patent
    - Updated scan/network_description.txt
    - Updated pom.xml

13. Miscellaneous script updates
    - Updated root-level: _adopt_replica_pvcs, _fix_replica_merlin, _import_pi,
      _patch_cluster, _prebind_pvcs, _rebind_d002, _rebind_d002b, test_resolve
    - Updated scripts/generate_spec.py

Co-authored-by: Junie <junie@jetbrains.com>
2026-03-22 22:16:21 -07:00
chrisfu
3f0f4b4c22 install.py drag and drop deploy. mv gcp to deploy dir. add init-port-fowards.sh and port-mappings.properties; now configured by Prole.app 2025-12-14 23:18:54 -08:00