Commit Graph

53 Commits

Author SHA1 Message Date
chrisfu
47ae5d69d6 fix(supabase/k3d): disable node affinity, ingress, and TUI clutter
- render_supabase.py: k3d sets scheduling.enforceGeneralNodeRole=false so
  pods don't require knoe.dev/node-role=general label (k3d nodes unlabelled)
- render_supabase.py: disable kong and studio ingress for k3d (port-forwards
  are used instead; avoids db.knoe.org reference that doesn't exist
- deploy.sh: escape double quotes in SQL comment on lines 2114-2115 to fix
  bash prematurely closing the psql -c ... string
- supabase.py: remove broken node-selector text entry (black rectangle in Tk)
  and redundant Deploy button (enable checkbox already triggers deployment)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
EOF
)
2026-05-03 22:15:07 -07:00
chrisfu
00f0ebec07 Merge claude/crazy-bose-fec256 into main
Bringing the long-running session-feature branch back into main in one
deliberate sweep. The branch carried the cluster work that's been live for
weeks (cross-cluster CNPG metrics, Grafana w/ Google OAuth, supabase
oauth2-proxy, cluster recovery, pg.0.knoe.dev + per-engineer onboarding,
GCS-backed CNPG backups via Workload Identity, the env-contamination
guard, the Junie brief queue, the cnpg-grafana CSRF + memory-request
fixes from today), while main accumulated Junie's parallel knoe-auth
Phase 2 OIDC work (full provider surface: discovery, authorize, token,
userinfo, JWKS, RS256 signing, code exchange, session services).

Key decision: the two branches did COMPETING rebrands off the same
starting point (5ba9b63, 2026-04-27):

  - claude branch (commit b355855, earlier): org.prole.authority.* →
                                              dev.knoe.auth.*
                                              (artifact renamed to
                                              knoe-auth.jar)
  - main (commit 9daa94b, recent): org.prole.authority.* →
                                    dev.knoe.authority.*
                                    (kept "authority" artifact name)

dev.knoe.auth wins: cluster runs from this name, the Maven artifact is
already knoe-auth.jar, and the broader rename is the documented
namespace direction (per ~/.claude/projects/-Users-chrisfu-dev-knoe-db/
memory/MEMORY.md). All of main's recent Phase 2 OIDC content was ported
from authority/src/.../dev/knoe/authority/ into
authority/src/.../dev/knoe/auth/ with package declarations rewritten.

== File-level resolution summary ==

Textual conflicts (4):

  authority/pom.xml
    - Took our artifactId="auth"
    - Took our branch's removal of spring-security-kerberos-client
      (verified: Junie's Phase 2 OIDC code does not import it; the dep
      was already-dead config)

  docs/pipeline-phases.md
    - Took our branch's "Phase 1 not started" status. Main had a
      misplaced " Complete" with a knoe-auth-Phase-1 commit ref
      in the autobuild Phase 1 section — different domain.

  docs/plans/knoe-auth-round-1.md
    - Took our branch's dev.knoe.auth file table (vs main's
      dev.knoe.authority listing). Pure rename mismatch.

  supabase/helm/knoe-supabase/templates/kong/config.yaml
    - Took our branch's onboard route + plain dashboard wiring.
      Main had an oauth2proxy.enabled toggle that put oauth2-proxy as
      a Kong upstream — but the deployed architecture (commit 25f1b2e)
      has oauth2-proxy in FRONT of Kong, not behind. Main's wrapper
      reflected an architecture that was never deployed.
    - Took our branch's removal of basic-auth from dashboard route
      (queue #15 brief still tracks the matching values.yaml /
      kong/deployment.yaml cleanup).

Java tree reconciliation (44 file-pairs):

  20 dual-path source files + 2 dual-path tests
    Body-identical between main's authority/ and our branch's auth/
    after stripping package decls — main's commit 9daa94b was a pure
    rebrand. Took our branch's auth/ version for all 22.

  8 main-only source files (Phase 2 OIDC), ported into auth/:
    web/JwksController.java
    web/OidcAuthorizeController.java
    web/OidcDiscoveryController.java
    web/OidcTokenController.java
    web/OidcUserInfoController.java
    session/OidcCodeService.java
    session/OidcTokenService.java
    session/SessionService.java

  12 main-only test files, ported into auth/:
    HealthControllerTest.java
    enroll/EnrollValueTypesTest.java
    enroll/EnrollmentControllerTest.java
    enroll/TotpServiceTest.java
    kerberos/KadminClientTest.java
    kerberos/KerberosSpnegoResultTest.java
    web/LoginControllerTest.java
    admin/AdminControllerTest.java
    user/PrincipalNormalizerTest.java
    regression/IdentityRegressionTest.java
    session/OidcCodeServiceTest.java
    session/SessionServiceTest.java

  Port mechanics: read main:authority/...<file> via git show, then sed
  rewrite `package dev.knoe.authority` → `package dev.knoe.auth` and
  `import dev.knoe.authority` → `import dev.knoe.auth`. Body content
  unchanged.

  authority/src/main/java/dev/knoe/authority/ — DELETED (duplicate)
  authority/src/test/java/dev/knoe/authority/  — DELETED (duplicate)

== Verification ==

- grep -rln '<<<<<<<' across .java/.md/.yaml/.yml/.sh/.xml/.tpl: clean
- find authority/src -path '*/dev/knoe/authority*': empty (subtree gone)
- grep 'package dev.knoe.authority' across repo: clean
- bash -n install.sh deploy.sh etc/preflight_kubecontext.sh: clean
- git ls-files -u | wc -l: 0 unmerged paths
- helm lint supabase/helm/knoe-supabase: pre-existing failure on
  studioIngress.enabled undefined in values.yaml (introduced by Junie
  on main; unrelated to this merge — flagging as follow-up).

== Followups (carried into TODO ranked queue or noted here) ==

  - helm lint failure: studioIngress block in values.yaml is missing
    enable flag; templates/studio/{ingress,oauth2proxy-deployment,
    oauth2proxy-service}.yaml all reference studioIngress.enabled with
    no default. Pre-existing on main; not introduced by this merge.
  - The five Junie briefs filed on this branch are now reachable from
    main at docs/plans/junie/{02,06,07,13,15}-*.md. Junie can pick them
    up in any order.
  - knoe-auth Phase 2 OIDC source (now at dev.knoe.auth.*) is not yet
    deployed to the cluster. Deployment is its own task.
  - The branch claude/crazy-bose-fec256 stays in place (worktree at
    .claude/worktrees/crazy-bose-fec256 may have ongoing context for
    Claude Code sessions). Safe to delete once next session starts
    cleanly from main.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-01 16:39:10 -07:00
chrisfu
55b6a6aff3 Complete rebranding from prole to knoe and fix macOS application identity. Bulk renamed 'prole' to 'knoe' across code, scripts, and manifests. Updated Makefile with 'knoe', 'build', and 'start' targets. Implemented macOS Application Bundle (.app) support for correct identity. Fixed macOS 'Python' process name to 'Knoe.DB Installer' via Objective-C bridge. Standardized application name to 'Knoe.DB Installer' across all interfaces.
Co-authored-by: Junie <junie@jetbrains.com>
2026-04-22 15:08:35 -07:00
chrisfu
391c4f5fc9 fix(net): BackendConfig healthchecks + externally-managed supabase-kong ingress
GCE L7 Ingresses for knoe-svc-kong (svc.knoe.dev / api.knoe.dev),
supabase-kong (api.0.knoe.dev) and supabase-studio (db.0.knoe.dev) were
all stuck UNHEALTHY: the default GCE healthCheck is HTTP GET `/` on the
backend port, but Kong returns 404 on any unrouted path and Studio
returns a 301 redirect -- neither passes the default probe, so the LB
serves "Server Error" instead of reaching the pod.

Replicate the pattern already working for gitlab-webservice-default:
emit a BackendConfig CRD with a TCP healthCheck on the service port and
annotate the Service with cloud.google.com/backend-config so GCE picks
it up. TCP is sufficient for LB-level liveness -- the backend is "alive"
as long as the process is accepting connections.

- etc/init_kong.sh: new SVC_KNOE_BACKEND_CONFIG_NAME; apply BackendConfig
  inside k8s/GCE branch; annotate Service post-apply.
- knoe-supabase chart: new kong/backendconfig.yaml + studio/backendconfig.yaml
  (TCP on 8000 / 3000), gated on service.{kong,studio}.backendConfigName.
- knoe-supabase chart: kong/service.yaml + studio/service.yaml pick up
  cloud.google.com/backend-config when backendConfigName is set.
- render_supabase.py: sets service.{kong,studio}.backendConfigName in k8s
  mode so the above wires up automatically.

Separately, the chart-managed supabase-kong Ingress was being reaped
from the cluster seconds after helm install (manifest present in the
release, gone via `kubectl get`). Root cause TBD -- suspected
meta.helm.sh/* annotation ownership colliding with a GKE/Anthos audit
controller. Workaround: render_supabase.py now emits a standalone
public-ingress-kong.yaml (no helm metadata) that supabase/deploy.sh
applies alongside public-ingress-tls.yaml, and the chart template gains
an `ingress.externallyManaged` guard so it no-ops in k8s mode. Default
`false` keeps k3d/k3s behavior unchanged.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-21 10:31:06 -07:00
chrisfu
ed163f0783 chore: add dependency checks and TLS resource handling for Supabase ingress
- Introduced dependency validation for ManagedCertificate and FrontendConfig annotations in Supabase ingresses.
- Enhanced `deploy.sh` with public ingress TLS resource application and improved error diagnostics.
- Updated `render_supabase.py` to generate GKE TLS manifests and log rendered resources.
- Refined Supabase deployment flow to include detailed ingress dependency validation and reconciliation.
2026-04-20 10:45:33 -07:00
chrisfu
8eff5dba0d chore: improve Supabase PVC diagnostics and ingress preflight validation
- Enhanced `deploy.sh` to include detailed storage class diagnostics for blocking PVCs (e.g., provisioner, type, binding mode).
- Added `validate_supabase_rendered_pvc_storage` to ensure rendered PVC configurations align with expected storage classes.
- Improved event parsing logic to prioritize recent PVC errors and resolve storage class details.
- Updated deployment flow to handle Supabase Studio ingress reconciliation errors if deployment is missing or replicas are zero.
2026-04-19 22:16:05 -07:00
chrisfu
c3b802c5f5 Fix Supabase stale PV cleanup enumeration path
Co-authored-by: Junie <junie@jetbrains.com>
2026-04-19 17:10:07 -07:00
chrisfu
2f3d424777 chore: enhance Supabase retained disk cleanup with project resolution and detailed skip reasons
- Updated `deploy.sh` to resolve project IDs from multiple configuration fallbacks and environment contexts for retained disk cleanup.
- Added support for `supabase_cfg_first_nonempty_value` to prioritize configuration over defaults.
- Enhanced logging with explicit reasons for skipped disk cleanup actions (e.g., missing gcloud, unresolved project).
- Updated tests to validate fallback logic and skip reason reporting.
2026-04-19 14:05:34 -07:00
chrisfu
be6ff2f1a8 chore: refactor Supabase PV cleanup logic and improve artifact handling
- Added `supabase_collect_stale_retained_pv_info` for clearer separation of logic and improved reusability.
- Enhanced zone detection for retained PVs using regex-based fallback.
- Updated cleanup logic to auto-enable `SUPABASE_AUTO_CLEAN_RETAINED_PVS` and streamline handling of GCE disk deletion.
- Introduced verification and retry mechanism to ensure stale PV artifacts are fully removed.
- Improved error handling for remaining GCE disk cleanup with detailed remediation instructions.
2026-04-18 20:55:29 -07:00
chrisfu
772e6a4047 chore: enhance Supabase PVC and ingress handling for deployment scripts
- Added detailed handling and cleanup for stale retained PVs to prevent quota exhaustion.
- Introduced `check_supabase_retained_pv_blocked` utility for preflight artifact detection and optional auto-cleanup.
- Improved ingress reconciliation for `supabase-kong` and `supabase-studio`.
- Enhanced retry logic with optional destructive namespace resets when the app cluster is unhealthy.
2026-04-18 20:35:19 -07:00
chrisfu
7a975e29b1 chore: scale down over-replicated GitLab deployments to target replica count of 1 2026-04-18 14:02:50 -07:00
chrisfu
b9047cd2ad chore: improve PVC event handling and enforce GitLab workload replica targets
- Added skipping logic for aged or bound PVC events in `deploy.sh` to reduce noise in diagnostics.
- Enforced replica target of 1 for specific GitLab workloads to ensure compliance with requirements.
2026-04-16 01:16:10 -07:00
chrisfu
d14605ea23 chore: extend PVC diagnostics and enhance StorageClass reconciliation
- Improved PVC error diagnostics in `deploy.sh` with additional context.
- Added robust StorageClass reconciliation logic, including dynamic creation and parameter validation.
- Updated Helm templates to support advanced StorageClass attributes like `reclaimPolicy` and `volumeBindingMode`.
2026-04-16 00:35:15 -07:00
chrisfu
5ebe4d6c06 chore: improve error logging and extend StorageClass validation
- Redirected error messages in `deploy.sh` and `render_supabase.py` to `stderr` for better logging clarity.
- Extended allowed prefixes for Supabase StorageClass to include `pd-standard`.
2026-04-16 00:05:11 -07:00
chrisfu
0f68c4ddfe chore: enforce strict StorageClass validation and add GitLab reconciliation logic
- Added hard validation for Supabase StorageClass configuration in `deploy.sh` and Helm templates.
- Implemented detailed reconciliation logic for GitLab workloads, addressing over-deployment scenarios.
- Enhanced `deploy.sh` with explicit namespace handling and PVC compliance checks.
2026-04-15 23:40:58 -07:00
chrisfu
d0fc4af23c chore: add Supabase PVC quota diagnostics and improve ingress handling
- Added PVC quota error detection with detailed logging in `deploy.sh`.
- Enhanced ingress readiness checks with additional Supabase ingress resources.
- Updated Helm templates to standardize and always enable ingress for API and Studio.
2026-04-15 23:06:51 -07:00
chrisfu
bdeb0b62f3 chore: add Supabase PVC quota diagnostics and improve ingress handling
- Added PVC quota error detection with detailed logging in `deploy.sh`.
- Enhanced ingress readiness checks with additional Supabase ingress resources.
- Updated Helm templates to standardize and always enable ingress for API and Studio.
2026-04-15 22:51:57 -07:00
chrisfu
06862bab5c chore: fix DB-to-APP context mismatches and refine deployment logic
- Resolved context mismatches by updating DB references to APP where appropriate.
- Updated Supabase deployment to ensure proper handling of cross-cluster DB hosts.
- Simplified ingress logic by setting `split_frontdoor_to_db` to `false` for APP clusters lacking GCE Ingress support.
2026-04-15 10:10:46 -07:00
chrisfu
16873885f5 chore: fix DB-to-APP context mismatches and refine deployment logic
- Resolved context mismatches by updating DB references to APP where appropriate.
- Updated Supabase deployment to ensure proper handling of cross-cluster DB hosts.
- Simplified ingress logic by setting `split_frontdoor_to_db` to `false` for APP clusters lacking GCE Ingress support.
2026-04-15 02:05:49 -07:00
chrisfu
f2d4e2adf2 chore: improve deployment logic and readiness checks
- Added automatic persistence disabling when deployment is disabled.
- Enhanced readiness checks with support for selectors and detailed status logging.
- Improved retry logic to avoid unnecessary namespace resets for healthy app clusters.
- Refined helm deployment flow with clearer readiness and health diagnostics.
2026-04-15 00:43:27 -07:00
chrisfu
8d911b2739 chore: add MinIO diagnostics and pod security configuration
- Introduced diagnostics for MinIO failures due to file access issues, with suggested fixes for pod security settings.
- Added podSecurityContext and securityContext configurations to enforce non-root execution for MinIO.
- Updated resource requests/limits for MinIO pods to ensure optimal resource usage.
- Standardized file path references in manifest-summary.json to relative paths.
2026-04-15 00:13:16 -07:00
chrisfu
174877201e chore: update GKE storage class and reconcile PVCs for Supabase deployment
- Changed default GKE non-DB PVCs to use CSI `pd-standard` with WFFC mode, deprecating legacy `standard`.
- Added reconciliation logic for APP PVCs to align with updated storage class.
- Enforced single replica for Supabase functions on GKE, disabling autoscaling.
- Updated tests to validate storage class changes, PVC reconciliation, and single-replica logic.
2026-04-14 11:13:14 -07:00
chrisfu
fddaf80823 checkpoint: make node placement config-driven and add GitLab rerun fast-path
- remove hardcoded physical host assumptions in UI/init flows and rely on config-driven selectors\n- optimize GitLab reruns to skip unnecessary operator upgrades/long reconcile waits when unchanged\n- add and update regression/shell tests for fast-path and k8s context/ingress/garage behaviors

Co-authored-by: Junie <junie@jetbrains.com>
2026-04-14 06:43:37 -07:00
chrisfu
492caabf60 chore: make node selector keys configurable and improve scheduling logic
- Refactored `init_gitlab.sh` and `init_gitea.sh` to use configurable node selector keys, removing hardcoded defaults like `gandalf.prole.org`.
- Enhanced scheduling logic to validate required fields and prevent stale node constraints during reruns.
- Added fast-path guards in GitLab init to skip redundant operations when no changes occur in operator or CR specifications.
- Updated Supabase deployment to clear stale topology spread constraints alongside node selectors and affinity.
- Added tests for configurable node selector keys, fast-path guards, and stale constraint cleanups.
2026-04-14 06:01:50 -07:00
chrisfu
9e10ffe8d7 chore: update Supabase deployment defaults and scheduling logic
- Changed default storage class for GKE to `standard` to reduce pressure on SSD/balanced quotas.
- Disabled `enforceGeneralNodeRole` scheduling for APP/DB values to support split cluster setups.
- Updated tests to validate storage class and scheduling logic changes.
2026-04-14 05:07:26 -07:00
chrisfu
9b9063a5b2 chore: add TLS configuration support for svc ingress and enhance readiness checks
- Introduced `SERVICE_INGRESS_TLS_ENABLED` for configurable svc ingress TLS rendering in `init_kong.sh`.
- Updated ingress templates to include or exclude TLS annotations and blocks dynamically.
- Improved PVC and pod readiness checks in `supabase/deploy.sh` with enriched logging and detailed blockers.
- Added support for configurable storage class resolution for Supabase DB frontdoor PVCs.
- Enhanced deployment workflows to handle split APP/DB cluster setups with context-aware readiness checks.
- Updated test cases to validate svc ingress TLS configuration and storage class logic.
2026-04-13 19:18:27 -07:00
chrisfu
9519f35d65 chore: add enforceGeneralNodeRole scheduling logic and improve frontdoor manifest handling
- Introduced `enforceGeneralNodeRole` to make node affinity and selector enforcement configurable.
- Updated `_split_frontdoor_docs` to handle custom release labels for Kong services.
- Enhanced Helm templates to support dynamic scheduling configurations.
- Added `reconcile_db_frontdoor_studio_pvcs` to migrate PVCs to the target storage class.
- Updated tests to validate custom release labels and scheduling logic.
2026-04-13 16:23:06 -07:00
chrisfu
35299fdd52 chore: add hostPath jemalloc optimization and enhance monitoring storage class handling
- Introduced jemalloc hostPath optimizations with configurable modes (`auto`, `off`, `force`).
- Integrated jemalloc setup with best-effort and forced validation flows for ensuring cluster compatibility.
- Enhanced monitoring storage class logic with mode-specific handling (`k3s`, `k3d`, `gke`) and improved validation of required classes.
- Added safeguards and detailed logging for unsupported configurations and failure scenarios.
2026-04-12 21:20:05 -07:00
chrisfu
e3c2e625f1 refactor(config): separate k3d k3s and gke config entrypoints
Rename env config files from conf/*/prole.cfg to conf/k3d.cfg, conf/k3s.cfg, and conf/gke.cfg. Update shell/Python loaders and etc/deploy scripts to resolve named configs cleanly while keeping legacy fallback behavior. Align k3s Ansible tasks, docs, and regression coverage with the new configuration layout.

Co-authored-by: Junie <junie@jetbrains.com>
2026-04-11 22:20:45 -07:00
chrisfu
c023c7f590 stabilize unattended GKE Supabase/CNPG deployment
- align app/db cluster defaults for knoe-dev-0 + knoe-dev-cnpg-0 on e2-standard-2\n- harden Supabase deploy flow for cross-cluster DB ILB, GKE storage class, and node selector fallback\n- migrate Helm chart path to knoe-supabase and make external DB host rendering IP-safe\n- disable Kerberos in prod unattended config to prevent Supabase GSS auth failures\n- add Supabase port-forward mappings and DB backup context handling improvements\n\nBuild status: k8s stable

Co-authored-by: Junie <junie@jetbrains.com>
2026-04-10 15:53:58 -07:00
chrisfu
df6de9138e checkpoint: summarize CNPG init and backup hardening work
- harden CNPG backup ObjectStore readiness fallback in statusless clusters by validating controller availability and expected spec

- improve backup/plugin reconciliation paths and related test coverage for barman-cloud flows

- include ongoing GKE/CNPG deployment and configuration updates from this checkpoint

Co-authored-by: Junie <junie@jetbrains.com>
2026-04-06 05:51:26 -07:00
chrisfu
93f7259722 chore: checkpoint k3s deployment and knoe updates
- update Kong/Supabase ingress and service manifests for k3s

- expand init scripts and runtime config/topology handling

- refresh KNØE UI screens and Supabase deployment/render logic

- include related env helper test adjustments

Co-authored-by: Junie <junie@jetbrains.com>
2026-03-31 23:15:54 -07:00
chrisfu
c65d336d1f Make init-script secret checks warning-only and unblock Next
Co-authored-by: Junie <junie@jetbrains.com>
2026-03-31 23:15:54 -07:00
chrisfu
65d03ed442 stable: slimmed Supabase + CNPG 3-node healthy build (2026-03-25)
STABLE BUILD — all 35+ pods Running, zero non-Running pods, CNPG ready=3.

## Architecture changes (Supabase slim-down)
- Removed local Supabase Kong, Storage, MinIO, Imgproxy from deployed stack
- Rewired API routing to shared Kong in kube-system
- Rewired object storage to shared Garage in knoe-system
- Supabase footprint reduced to control-plane only: auth, rest, meta,
  analytics, realtime (studio port-forward retained)
- Supabase pods renamed: stripped redundant '-prole-supabase-' prefix via
  per-component fullnameOverride (e.g. supabase-auth, supabase-rest, ...)
- Supabase workloads pinned to merlin.prole.org (memory-safe node)

## File changes
- conf/service/prole.cfg
    * Replaced all hardcoded /Users/chrisfu/dev/prole paths with ${HOME}/dev/prole
      so config is portable when run from myrddin.prole.org or any user home
    * SUPABASE_ENABLED=True, SUPABASE_PRIMARY_NODE=merlin.prole.org in
      both [Global] and [Optional Features]
    * supabase_enabled=true in [Inputs]/init_cluster section
- supabase/helm/render_supabase.py
    * Explicitly disabled: kong, storage, minio, imgproxy
    * Active components pinned via nodeSelector to merlin.prole.org
    * fullnameOverride per component to clean up pod names
    * Default primary node changed from pi.prole.org -> merlin.prole.org
- supabase/deploy.sh
    * Removed svc/kong port-forward; API via shared Kong in kube-system
    * Removed imgproxy/storage from PV/PVC prep and helm --set persistence args
    * Fixed kubectl exec missing -i flag for psql heredoc stdin in
      setup_knoe_db_for_supabase
    * Updated Studio access messaging to reference shared platform Kong
- k8s/prole/knoe-db.yaml
    * instances: 3
    * Added control-plane/master tolerations for myrddin scheduling
- deploy/opentofu/k3s/manifests/prole/knoe-db.yaml
    * instances: 3, added control-plane tolerations (mirrors k8s/prole/)
- k8s/prole/iscsi-pvs.yaml
    * Expanded d003-data/d003-wal nodeAffinity to [pi, merlin, myrddin]
      to reflect post-OOM-recovery live state
- k8s/registry/deployment.yaml
    * nodeSelector restored to myrddin.prole.org (hostPort 5000 owner)
    * Added control-plane + master tolerations so pod schedules correctly
- modes/k3s/knoe-db/.version
    * Aligned to 140 (image tag 18-140) matching repo-root version
- etc/init_common_services.sh, etc/init_registry.sh
    * Minor fixes aligned with slimmed service layout and node targeting
- infrastructure/inventory/host_vars/pi.prole.org.yml
    * Updated host vars to reflect post-recovery pi state
- knoe/core/topology.py, knoe/ui/screens/cluster.py,
  knoe/ui/screens/cluster_nodes.py
    * Topology and UI updates for 3-node cluster representation
- scan/network_description.txt
    * Refreshed network scan output post-recovery
- conf/port-mapping.cfg
    * Updated port mapping to reflect removal of local Kong/storage forwards
- scripts/cleanup_stale_rs.sh (new)
    * Utility script to prune orphaned ReplicaSets after rollouts

## Verified stable state
- CNPG: Cluster in healthy state, ready=3
    knoe-db-2 (merlin, primary), knoe-db-3 (pi), knoe-db-5 (myrddin)
- Supabase: 5 pods Running on merlin.prole.org
    supabase-auth, supabase-rest, supabase-meta,
    supabase-analytics, supabase-realtime
- Monitoring: all prometheus/grafana/alertmanager pods Running on merlin
- Registry: Running on myrddin.prole.org (hostPort 5000)
- Zero non-Running / non-Completed pods across all namespaces

Co-authored-by: Junie <junie@jetbrains.com>
2026-03-25 11:56:40 -07:00
chrisfu
9ad1054a55 STABLE BUILD: All 9 Supabase pods Running on merlin, db.prole.org live (HTTP 401)
Fix supabase-storage migration deadlock: ALTER ROLE search_path + BYPASSRLS

Root cause: supabase_storage_admin had no rolconfig search_path, so the connection
string search_path=auth caused the service to query auth.migrations (not found →
currentMigrations=[]) but insert migration tracking rows into public.migrations
(via PostgreSQL implicit public fallback). On restart, currentMigrations was always
[] despite public.migrations having rows, causing duplicate-key on migration 0.

Fix (added to deploy.sh db init block, idempotent on every reinstall):
  ALTER ROLE supabase_storage_admin SET search_path = storage, auth, public;
  ALTER ROLE supabase_storage_admin BYPASSRLS;
  TRUNCATE TABLE IF EXISTS public.migrations;
  DROP TABLE IF EXISTS storage.migrations CASCADE;

This aligns migration reads and writes to the same schema (storage first, then auth,
then public) and ensures a clean slate on every redeploy.

Result: all 9 Supabase pods Running on merlin.prole.org (merlin-local-iscsi-d002 iSCSI
storage), db.prole.org returns HTTP 401 (Supabase Studio auth gate = live and working),
CNPG knoe-db 2/2 Ready on merlin + myrddin with Percona PostgreSQL 18 + Barman backup.

Co-authored-by: Junie <junie@jetbrains.com>
2026-03-24 15:19:10 -07:00
chrisfu
e0616fda3f STABLE BUILD: Supabase running on merlin.prole.org with CNPG backend (knoe-db)
All 9 Supabase pods Running on merlin.prole.org backed by merlin-local-iscsi-d002
(/synology/d002) iSCSI storage. db.prole.org (HTTP 401 = auth gate = reachable).
CNPG cluster knoe-db 2/2 Ready (merlin + myrddin), Percona PostgreSQL 18, Barman backup.

Key fixes in this build:
- supabase_enabled=true now honored in silent mode (actions.py override removed)
- SUPABASE_STORAGE_CLASS/PV_NODE/PRIMARY_NODE pinned to merlin in prole.cfg
- _write_cfg preserves SUPABASE_* vars from existing cfg (no pipeline write-back clobber)
- render_supabase.py writes persistence.*.storageClassName (correct Helm chart path)
- deploy.sh: synology guard prevents non-iSCSI storage; --set persistence.* all modes;
  stale values.generated.json deleted before every render; live DB_PASSWORD from secret
- init_monitoring.sh: pv_prom/pv_am/pv_graf derived inline before Released-PV loop
- k8s/prole/knoe-db.yaml: instances=2 (pi excluded), maxSyncReplicas=0
- Static PVs merlin-supabase-{functions,imgproxy,snippets,storage} created on merlin
- storage.migrations DROP fixes idempotent reinstall conflict from prior deploy

Co-authored-by: Junie <junie@jetbrains.com>
2026-03-24 14:59:19 -07:00
chrisfu
6f3f62f829 Fix Supabase storage class injection: write persistence.*.storageClassName, add synology guard
Root cause: render_supabase.py wrote to deployment.*.storageClass (ignored by Helm chart PVC
templates) instead of persistence.*.storageClassName (_pvc.tpl:43 is the real path). Also
deploy.sh --set persistence.* args only ran in k8s mode, and stale values.generated.json
was reused across deploys. Together these caused Supabase PVCs to fall through to
pi-local-iscsi (pi SD card), crashing the node under memory pressure.

- supabase/helm/render_supabase.py:
  - write persistence.{functions,imgproxy,snippets,storage,deno,minio}.storageClassName
    using the correct Helm chart path (_pvc.tpl:43)
  - add synology guard: raise SystemExit if storage class is not synology/merlin/myrddin
  - remove broken deployment.*.storageClass writes
- supabase/deploy.sh:
  - delete stale values.generated.json before every helm_render_values call
  - fall back to reading SUPABASE_STORAGE_CLASS from cfg file via python3 if env not set
  - add bash synology guard: die if storage class is not iSCSI/NFS mounted
  - extend helm_set_args to include persistence.minio.storageClassName and apply
    --set persistence.* for ALL modes (not just k8s)

Co-authored-by: Junie <junie@jetbrains.com>
2026-03-24 14:27:15 -07:00
chrisfu
6f29deba99 Supabase/CNPG stability: pin Supabase to merlin, reduce CNPG to 2 instances, fix supabase_enabled
Changes address resource pressure (merlin OOM with 3 CNPG + Supabase + monitoring) and
the supabase_enabled=false override in silent mode that blocked Supabase deploy.

- conf/service/prole.cfg:
  - SUPABASE_PV_NODE/SUPABASE_PRIMARY_NODE = merlin.prole.org (d002 storage)
  - SUPABASE_STORAGE_CLASS = merlin-local-iscsi-d002
  - CNPG_ELIGIBLE_NODES: removed pi.prole.org (pi NotReady, d003 inaccessible)
- k8s/prole/knoe-db.yaml:
  - instances: 3 -> 2 (merlin + myrddin only, pi excluded)
  - maxSyncReplicas: 1 -> 0 (safe for 2-node cluster)
- supabase/deploy.sh:
  - default SUPABASE_PV_NODE to merlin.prole.org, base dir to /synology/d002/supabase
  - add idempotent ALTER ROLE password sync after CREATE ROLE IF NOT EXISTS
  - resolve live DB_PASSWORD from knoe-db-superuser secret before helm render
- supabase/helm/render_supabase.py:
  - resolve SUPABASE_STORAGE_CLASS from env/cfg, apply to storage/analytics PVCs
  - DATABASE_NAMESPACE resolution: check env + cfg Global section before discovery
  - DB_PASSWORD from env takes priority over cfg (live cluster secret wins)
  - default primary node = merlin.prole.org
- knoe/ui/screens/cluster.py: minor display update

Co-authored-by: Junie <junie@jetbrains.com>
2026-03-24 13:52:38 -07:00
chrisfu
498a7f0412 Migrate Supabase to pi.prole.org, fix supabase_enabled override, monitoring PV vars
Memory pressure on merlin (95% OOM with monitoring + CNPG replica + Supabase) resolved
by moving Supabase workload to pi.prole.org which has headroom (49% memory, 2 pods).

- conf/service/prole.cfg: set SUPABASE_PV_NODE, SUPABASE_PRIMARY_NODE=pi.prole.org,
  SUPABASE_STORAGE_CLASS=pi-local-iscsi, SUPABASE_PV_BASE_DIR for rancher local-path
- supabase/helm/render_supabase.py: resolve SUPABASE_STORAGE_CLASS from env/cfg,
  apply pi-local-iscsi to storage/analytics PVCs, resolve DATABASE_NAMESPACE for
  correct db_host, DB_PASSWORD env override takes priority for live cluster password
- supabase/deploy.sh: default SUPABASE_PV_NODE to pi.prole.org, default base dir to
  /var/lib/rancher/k3s/storage/supabase, add idempotent ALTER ROLE password sync,
  live DB_PASSWORD resolution in helm_render_values from knoe-db-superuser secret
- knoe/core/actions.py: stop overriding init_cluster.supabase_enabled to False in
  silent mode so prole.cfg value is honored end-to-end
- etc/init_monitoring.sh: derive pv_prom/pv_am/pv_graf inline before Released-PV
  loop to fix unbound variable error on idempotent monitoring installs

Co-authored-by: Junie <junie@jetbrains.com>
2026-03-24 12:01:02 -07:00
chrisfu
5e7c03324e Fix Supabase deploy: honor supabase_enabled, pin dirprep to merlin (d002), fix monitoring PV vars
- actions.py: stop overriding init_cluster.supabase_enabled to False in silent mode
- supabase/deploy.sh: change SUPABASE_PV_NODE default from myrddin to merlin.prole.org
  so prole-supabase-dirprep job runs on the node that owns /synology/d002
- conf/service/prole.cfg: add SUPABASE_PV_NODE and SUPABASE_PV_BASE_DIR to [Global]
  so prole_cfg.sh exports them explicitly on every run
- init_monitoring.sh: derive pv_prom/pv_am/pv_graf inline before Released-PV loop
  to fix unbound variable error on idempotent monitoring installs

Co-authored-by: Junie <junie@jetbrains.com>
2026-03-24 00:16:26 -07:00
chrisfu
a069989315 Rename prole-db to knoe-db, add knoe-auth as cluster-internal KDC
Itemized changes:

1. knoe-auth: New cluster-internal KDC and SSO gateway service
   - Created etc/init_knoe_auth.sh based on init_kdc.sh with knoe-auth naming
   - Namespace defaults to SERVICE_NAMESPACE (knoe-system)
   - ConfigMap: knoe-auth-kdc-config, Secret: knoe-auth-secrets
   - Legacy cleanup removes old auth/dog/authority deployments

2. Orchestration: knoe-auth initializes before CloudNativePG
   - Updated prole.sh to insert init_knoe_auth.sh as step 2 (before CNPG)
   - Renumbered all subsequent initialization steps

3. Kong routing: Updated init_kong.sh to route to knoe-auth in SERVICE_NAMESPACE

4. Comment/reference updates for knoe-auth
   - Updated init_common_services.sh, init_service_layer.sh, init_kerberos.sh

5. prole-db renamed to knoe-db across the entire codebase
   - Renamed prole-db/ directory to knoe-db/
   - Renamed all prole-db Kubernetes manifests (deploy/opentofu, k8s/)
   - Renamed scripts: docker-root-knoe-db.sh, docker-run-knoe-db.sh, test-cnpg-knoe-db.sh
   - Renamed etc/init_prole-db-reset.sh to etc/init_knoe-db-reset.sh
   - Renamed etc/prole-db-passwwd.sh to etc/knoe-db-passwwd.sh
   - Renamed mock_val counterparts accordingly
   - Renamed tests/etc/test_init_prole-db-reset.sh to test_init_knoe-db-reset.sh
   - Renamed docs/prole-db-documentation-mcp-architecture.md to knoe-db variant
   - Renamed modes/k3d/prole-db/ to modes/k3d/knoe-db/
   - Renamed prole-db.iml to knoe-db.iml

6. Configuration updates
   - Updated conf/dev, conf/prod, conf/test, conf/service prole.cfg files
   - Updated conf/port-mapping.cfg
   - Updated etc/prole_cfg.sh and mock_val/prole_cfg.sh
   - Updated service/prole.cfg

7. Kubernetes manifests and deploy configuration
   - Updated deploy/opentofu/k3s ArgoCD application YAMLs
   - Updated kong-configmap.yaml and kustomization.yaml
   - Updated k3s/kong-config.yml and prole-resources.yaml
   - Updated prole-mssql-db deployment YAMLs
   - Updated supabase helm render and deploy scripts

8. Infrastructure and GCP Terraform
   - Updated deploy/gcp/terraform: folders, groups, IAM, service-projects

9. Python/installer code updates
   - Updated knoe/core: actions, build_context, controller, env, milestones
   - Updated knoe/milestone.py
   - Updated knoe/ui/screens: cfg, database, database_options, deploy, docker,
     navigation, security, services, validate
   - Updated knoe.spec, status.py

10. Shell script updates
    - Updated etc/: build_db, init_cloudnative_pg, init_cnpg_backup,
      init_db_manager, init_forgejo, init_gitlab, init_monitoring, init_openbao,
      init_port_forwards, init_postgrest, init_supabase_ports, status
    - Updated mock_val/ counterparts for all above scripts
    - Updated prole-net/init-prole-dns.sh
    - Updated bin/prole-kpf.sh, gitea/deploy.sh, supabase/deploy.sh

11. Test updates
    - Updated tests/etc/: test_init_cloudnative_pg*, test_init_cnpg_backup*,
      test_init_kdc*, test_init_kerberos*, test_init_kong*, test_prole_cfg*
    - Updated tests/installer/: test_actions_helpers, test_cfg_save_kubecontext,
      test_controller, test_core_classes, test_milestones, test_milestones_extended,
      test_namespace_propagation
    - Updated tests/: test_database_options, test_navigation,
      test_render_supabase_hostname, test_docker_build_fix,
      test_all_prole_home_fixes, silent_install_test, final_test

12. Documentation updates
    - Updated docs/: DOCKER-BUILD-FIX, PROLE-CFG-SECRETS, PROLE-HOME-DIRECTORY,
      build-system, patent
    - Updated scan/network_description.txt
    - Updated pom.xml

13. Miscellaneous script updates
    - Updated root-level: _adopt_replica_pvcs, _fix_replica_merlin, _import_pi,
      _patch_cluster, _prebind_pvcs, _rebind_d002, _rebind_d002b, test_resolve
    - Updated scripts/generate_spec.py

Co-authored-by: Junie <junie@jetbrains.com>
2026-03-22 22:16:21 -07:00
chrisfu
9b9d6fdc88 Rename Prole storage objects to Synology and sync deployment updates
- Rename iSCSI storage class and PV/PVC selectors/labels from prole to synology across k8s and OpenTofu manifests\n- Update CNPG/OpenBao/Garage/monitoring init flows, render helpers, and mock scripts for synology-backed storage objects\n- Integrate related UI/core/service config/version updates and add supporting regression tests for CNPG storage/image behavior\n- Keep storage reconciliation tests aligned with current CNPG affinity output

Co-authored-by: Junie <junie@jetbrains.com>
2026-03-22 21:50:02 -07:00
chrisfu
133b719479 Align CNPG bootstrap placement with Ansible policy
Prefer infra-managed manifests for bootstrap; reconcile CNPG instances based on Ready+schedulable labeled db nodes; update manifests to use node-role affinity + anti-affinity; add policy/tests and config touch-ups (incl. prole.cfg).
2026-03-18 22:06:41 -07:00
chrisfu
ce5ff83eb4 checkpoint: improve init scripts, installer flows, and kube context handling
- Fix kube context switching for k3s single-context kubeconfigs and k3d shorthand prefixes

- Update common init/status scripts (registry, kerberos, cnpg backup, service layer, common services)

- Add Gitea init script and installer ArgoCD screen

- Add Supabase realtime probe patching plus regression tests

- Extend installer core/UI test coverage
2026-03-14 20:07:54 -07:00
chrisfu
6dc3560dcf refactor: rename backup script to etc/init_cnpg_backup.sh and optimize Supabase image prefetch
- Renamed etc/init_prole-db-backup.sh to etc/init_cnpg_backup.sh and updated all references.

- Enhanced CNPG backup script with authoritative status validation.

- Implemented SupabaseImagePreloadMilestone to front-load heavy image transfers.

- Updated supabase/deploy.sh with --prefetch-images-only and --skip-prefetch.

- Updated unit tests for CNPG backups and Supabase silent milestone sequencing.

- Included incidental environment updates from installer execution.

Co-authored-by: Junie <junie@jetbrains.com>
2026-03-02 00:34:01 -08:00
chrisfu
5e0a1bda85 feat: Add GitOps (Gitea) and Supabase integration, plus database options
- Makefile: Added 'init' and 'deploy' targets for k3s parity and Gitea staging.

- OpenTofu: Fixed namespace handling in k3s main.tf to prevent metadata overwrites.

- UI: Added 'GitOps' and 'Database Options' configuration screens.

- Core: Enhanced monitoring, milestones, and environment handling for new services.

- Supabase: Integrated full Helm chart and manifest rendering logic.

- Gitea: Added deployment scripts and GitOps sync support.

- Database: Added Percona/Postgres Dockerfile templates and improved TDE scripts.

- Tests: Added coverage for new UI screens and navigation flows.
2026-02-26 18:32:15 -08:00
chrisfu
f589cbd397 Remove PostgREST deployment, service, and related configs from prole-db namespace
- Deleted PostgREST Kubernetes deployment and service manifests.
- Removed `init_postgrest.sh` initialization script and PostgREST references across UI, installer, and config.
- Updated Kong API Gateway to exclude `/rest/*` routing to PostgREST.
- Simplified `supabase/deploy.sh` by removing references to PostgREST scaling and alias creation.
- Adjusted cluster configuration and UI to reflect the absence of PostgREST.
2026-02-25 00:38:07 -08:00
chrisfu
1573bb59c2 Add prole.spec and extend OpenTofu k3s configuration
- Include new `prole.spec` for build configurations and dependencies.
- Add Terraform state handling for OpenTofu in `k3s` cluster.
- Provision multiple Kubernetes resources in `prole-db` namespace: namespace, services, ConfigMaps, StatefulSets, Ingress rules, and PersistentVolumes.
- Integrate deployment and configuration enhancements for `garage`, `prole`, and related components.
2026-02-24 21:47:02 -08:00
chrisfu
b03efa8f69 Kong API gateway, docker-import preload, OpenTofu graceful fallback, milestone fix
Kong API Gateway (replacing prole nginx):
- Add etc/init_kong.sh provisioning script (DB-less mode, prole-db namespace)
- Add kong-deployment.yaml and kong-service.yaml manifests
- Rewire ingress rules (svc/git/api.prole.org) to prole-db-kong:8000
- Update kustomization.yaml to reference kong manifests

PostgREST & DB Manager in prole-db namespace:
- Add etc/init_postgrest.sh and etc/init_db_manager.sh scripts
- Add postgrest/db-manager deployment and service manifests
- Add src/db-manager/ Node.js REST endpoint for backup triggers
- Default NAMESPACE changed to prole-db in both scripts

Docker image pre-load from PROLE_DATA/docker-import:
- Add _preload_docker_images() to init_common_services.sh
- Scan for .tar files exported by final_deployment.sh
- Import via k3d image import (k3d) or ctr (k3s) before deployments
- Increase rollout timeouts to 300s (configurable via ROLLOUT_TIMEOUT) in init_openbao.sh, init_opentofu.sh, init_garage_store.sh, init_registry.sh

OpenTofu password resolution fix:
- Add Kubernetes secret fallback in resolve_admin_password()
- Change hard exit 1 to graceful return 1 with warning
- Wrap call in if-guard so set -e doesn't abort the script chain

Milestone fix (init scripts not running):
- Add init_kong.sh, init_postgrest.sh, init_db_manager.sh to InitializationScriptsMilestone.execute() script list and arg branches
- Previously only actions.py had these; milestones.py was missing them

Installer integration:
- Add Kong/PostgREST/DB Manager to silent installer _step_init_scripts
- Add corresponding tabs and execution blocks in UI services.py
2026-02-22 00:57:49 -08:00
chrisfu
f2c9012cce Refactor installation and initialization logic, and expand test coverage
- install.py: Major update including configuration variable expansion, improved k3s/k3d handling, and enhanced installation logic.

- etc/ scripts: Significant refactoring of initialization scripts (Kerberos, Port Forwards, Garage Store, etc.).

- Port Forwards: Transitioned from XML to port-mappings.conf for managing kubectl port-forwards.

- Status Reporting: Improved status checking for common services.

- Infrastructure: Updated Ansible inventory and rsyslog role configurations.

- Tests: Added a comprehensive suite of tests for 'etc' initialization scripts in prole/tests/etc/.

- Documentation: Added prole-db-documentation-mcp-architecture.md.

- General: Updated Dockerfiles and various helper scripts.
2026-02-13 21:36:39 -08:00