Commit Graph

14 Commits

Author SHA1 Message Date
chrisfu
b1755f4e7b Stabilize prod deployment flow and harden Garage/CNPG ops
Switch production config to k8s/GKE contexts and align service naming. Add immutable StatefulSet update fallback for Garage across k3d/k3s/k8s. Harden CNPG deploy and backup bootstrap paths, and update installer coverage for CNPG webhook and Garage common ops.

Co-authored-by: Junie <junie@jetbrains.com>
2026-04-11 20:19:36 -07:00
chrisfu
c023c7f590 stabilize unattended GKE Supabase/CNPG deployment
- align app/db cluster defaults for knoe-dev-0 + knoe-dev-cnpg-0 on e2-standard-2\n- harden Supabase deploy flow for cross-cluster DB ILB, GKE storage class, and node selector fallback\n- migrate Helm chart path to knoe-supabase and make external DB host rendering IP-safe\n- disable Kerberos in prod unattended config to prevent Supabase GSS auth failures\n- add Supabase port-forward mappings and DB backup context handling improvements\n\nBuild status: k8s stable

Co-authored-by: Junie <junie@jetbrains.com>
2026-04-10 15:53:58 -07:00
chrisfu
df6de9138e checkpoint: summarize CNPG init and backup hardening work
- harden CNPG backup ObjectStore readiness fallback in statusless clusters by validating controller availability and expected spec

- improve backup/plugin reconciliation paths and related test coverage for barman-cloud flows

- include ongoing GKE/CNPG deployment and configuration updates from this checkpoint

Co-authored-by: Junie <junie@jetbrains.com>
2026-04-06 05:51:26 -07:00
chrisfu
15890187a3 Stabilize CNPG reset/update flow and finalize 3-node recovery
- add idempotent CNPG operator reconciliation to prevent duplicate active controller ReplicaSets
- restore and validate CNPG 3-instance convergence with PV claimRef recovery and storage path readiness
- wire canonical launcher aliases/reset behavior and improve namespace cleanup semantics
- harden backup/objectstore readiness handling and retry behavior
- enforce service namespace usage for common services and remove default-namespace drift
- enable Kerberos milestone auto-activation when realm/kdc are configured and apply CNPG GSS pg_hba rules
- keep final runtime healthy: knoe-db 3/3, operator stable, perfsnap captured

Co-authored-by: Junie <junie@jetbrains.com>
2026-03-27 21:54:53 -07:00
chrisfu
a1177d71bc Add GitOps provider choice screen and cluster UI improvements
- Add _render_gitops_choice_page() to GitOpsScreenMixin for explicit
  Gitea / ArgoCD / None provider selection with radio buttons
- Extend cluster screen with GitOps provider radio group and canvas layout
- Refactor services screen layout and navigation registration
- Register new gitops choice screen in screens __init__ / navigation
- Expand Kong init scripts (etc/ and mock_val/) with additional logic
- Update init_cnpg_backup.sh with minor fix
- Refresh conf/service and conf/prod prole.cfg generated configs
- Update conf/port-mapping.cfg port entries
- Update Supabase Helm values.yaml
- Bump modes/k3s/knoe-db/.version
- Update network scan description
- Add/expand tests: test_cluster_screen_layout, test_gitops_choice_screen

Co-authored-by: Junie <junie@jetbrains.com>
2026-03-24 22:12:25 -07:00
chrisfu
801a5e3b74 Fix idempotent k3s deploy: TLS/DB secret bootstrap, PV release, barman idempotency, kubeconfig priority
Co-authored-by: Junie <junie@jetbrains.com>
2026-03-23 23:31:19 -07:00
chrisfu
a069989315 Rename prole-db to knoe-db, add knoe-auth as cluster-internal KDC
Itemized changes:

1. knoe-auth: New cluster-internal KDC and SSO gateway service
   - Created etc/init_knoe_auth.sh based on init_kdc.sh with knoe-auth naming
   - Namespace defaults to SERVICE_NAMESPACE (knoe-system)
   - ConfigMap: knoe-auth-kdc-config, Secret: knoe-auth-secrets
   - Legacy cleanup removes old auth/dog/authority deployments

2. Orchestration: knoe-auth initializes before CloudNativePG
   - Updated prole.sh to insert init_knoe_auth.sh as step 2 (before CNPG)
   - Renumbered all subsequent initialization steps

3. Kong routing: Updated init_kong.sh to route to knoe-auth in SERVICE_NAMESPACE

4. Comment/reference updates for knoe-auth
   - Updated init_common_services.sh, init_service_layer.sh, init_kerberos.sh

5. prole-db renamed to knoe-db across the entire codebase
   - Renamed prole-db/ directory to knoe-db/
   - Renamed all prole-db Kubernetes manifests (deploy/opentofu, k8s/)
   - Renamed scripts: docker-root-knoe-db.sh, docker-run-knoe-db.sh, test-cnpg-knoe-db.sh
   - Renamed etc/init_prole-db-reset.sh to etc/init_knoe-db-reset.sh
   - Renamed etc/prole-db-passwwd.sh to etc/knoe-db-passwwd.sh
   - Renamed mock_val counterparts accordingly
   - Renamed tests/etc/test_init_prole-db-reset.sh to test_init_knoe-db-reset.sh
   - Renamed docs/prole-db-documentation-mcp-architecture.md to knoe-db variant
   - Renamed modes/k3d/prole-db/ to modes/k3d/knoe-db/
   - Renamed prole-db.iml to knoe-db.iml

6. Configuration updates
   - Updated conf/dev, conf/prod, conf/test, conf/service prole.cfg files
   - Updated conf/port-mapping.cfg
   - Updated etc/prole_cfg.sh and mock_val/prole_cfg.sh
   - Updated service/prole.cfg

7. Kubernetes manifests and deploy configuration
   - Updated deploy/opentofu/k3s ArgoCD application YAMLs
   - Updated kong-configmap.yaml and kustomization.yaml
   - Updated k3s/kong-config.yml and prole-resources.yaml
   - Updated prole-mssql-db deployment YAMLs
   - Updated supabase helm render and deploy scripts

8. Infrastructure and GCP Terraform
   - Updated deploy/gcp/terraform: folders, groups, IAM, service-projects

9. Python/installer code updates
   - Updated knoe/core: actions, build_context, controller, env, milestones
   - Updated knoe/milestone.py
   - Updated knoe/ui/screens: cfg, database, database_options, deploy, docker,
     navigation, security, services, validate
   - Updated knoe.spec, status.py

10. Shell script updates
    - Updated etc/: build_db, init_cloudnative_pg, init_cnpg_backup,
      init_db_manager, init_forgejo, init_gitlab, init_monitoring, init_openbao,
      init_port_forwards, init_postgrest, init_supabase_ports, status
    - Updated mock_val/ counterparts for all above scripts
    - Updated prole-net/init-prole-dns.sh
    - Updated bin/prole-kpf.sh, gitea/deploy.sh, supabase/deploy.sh

11. Test updates
    - Updated tests/etc/: test_init_cloudnative_pg*, test_init_cnpg_backup*,
      test_init_kdc*, test_init_kerberos*, test_init_kong*, test_prole_cfg*
    - Updated tests/installer/: test_actions_helpers, test_cfg_save_kubecontext,
      test_controller, test_core_classes, test_milestones, test_milestones_extended,
      test_namespace_propagation
    - Updated tests/: test_database_options, test_navigation,
      test_render_supabase_hostname, test_docker_build_fix,
      test_all_prole_home_fixes, silent_install_test, final_test

12. Documentation updates
    - Updated docs/: DOCKER-BUILD-FIX, PROLE-CFG-SECRETS, PROLE-HOME-DIRECTORY,
      build-system, patent
    - Updated scan/network_description.txt
    - Updated pom.xml

13. Miscellaneous script updates
    - Updated root-level: _adopt_replica_pvcs, _fix_replica_merlin, _import_pi,
      _patch_cluster, _prebind_pvcs, _rebind_d002, _rebind_d002b, test_resolve
    - Updated scripts/generate_spec.py

Co-authored-by: Junie <junie@jetbrains.com>
2026-03-22 22:16:21 -07:00
chrisfu
2244d6acd6 Refactor iSCSI Synology mounts to /synology/d00x
- Rename host mount namespace from /prole/d00x to /synology/d00x

- Make LUN ownership explicit per host in inventory (myrddin=d001; merlin=d002,d004; pi=d003)

- Add safe migration cleanup for legacy /prole/d00x mounts and stale /etc/fstab entries

- Update k3s/ArgoCD/OpenTofu manifests and PV node affinities to match new mount paths

- Improve iSCSI role check-mode behavior and add quiesce/detach flow in ansible.sh
2026-03-21 10:43:06 -07:00
chrisfu
ecacf01266 Remove legacy secrets, configs, and scripts; introduce prole-auth service and Grafana SSO proxy configuration. 2026-03-20 13:00:36 -07:00
chrisfu
a445c77233 checkpoint: installer env var expansion + k3s registry + CNPG backups
- Expand nested $VAR/${VAR} (and ~) when resolving PROLE_* paths in installer core/UI to avoid literal $HOME dirs\n- Improve k3s registry host:port derivation (explicit cfg/env, server URL, kubeconfig fallback) and registry status UX\n- Update CNPG backup/init scripts for plugin-based backups + ScheduledBackup; improve plugin socket readiness and status output\n- Add/extend tests covering env var expansion, registry hostport resolution, and CNPG backup behavior
2026-03-14 22:55:32 -07:00
chrisfu
ce5ff83eb4 checkpoint: improve init scripts, installer flows, and kube context handling
- Fix kube context switching for k3s single-context kubeconfigs and k3d shorthand prefixes

- Update common init/status scripts (registry, kerberos, cnpg backup, service layer, common services)

- Add Gitea init script and installer ArgoCD screen

- Add Supabase realtime probe patching plus regression tests

- Extend installer core/UI test coverage
2026-03-14 20:07:54 -07:00
chrisfu
bb2135b587 fix(cnpg): converge to 3 instances with resilient apply
- Wait for Kubernetes API readiness before applying CNPG resources

- Retry kubectl apply on transient OpenAPI/webhook failures

- Enforce spec.instances=3 by patching existing Cluster if needed

- Wait for CNPG webhook endpoints before creating Backup resources

- Update bash tests/mocks to cover readiness + retry behavior

Co-authored-by: Junie <junie@jetbrains.com>
2026-03-12 01:42:42 -07:00
chrisfu
c0c7c5539c Refactor namespace handling to use prole.cfg as sole source of truth
- Removed all logic that read or set NAMESPACE/PROLE_NAMESPACE from the shell environment or wrote it to env.sh.

- prole.cfg is now the sole source of truth for the namespace value, loaded exclusively by prole_cfg.sh.

- etc/prole_cfg.sh & mock_val/prole_cfg.sh: Removed kubectl-context fallback and default for PROLE_NAMESPACE.

- Shell scripts (init_*.sh): Replaced NAMESPACE=${NAMESPACE:-...} with NAMESPACE="${PROLE_NAMESPACE}".

- Python (actions, environment, milestone): Removed env["PROLE_NAMESPACE"] from subprocess env dicts and env.sh.

- etc/init_cloudnative_pg.sh: Removed DB_PASSWORD env fallback for secret creation.

Co-authored-by: Junie <junie@jetbrains.com>
2026-03-02 18:50:53 -08:00
chrisfu
6dc3560dcf refactor: rename backup script to etc/init_cnpg_backup.sh and optimize Supabase image prefetch
- Renamed etc/init_prole-db-backup.sh to etc/init_cnpg_backup.sh and updated all references.

- Enhanced CNPG backup script with authoritative status validation.

- Implemented SupabaseImagePreloadMilestone to front-load heavy image transfers.

- Updated supabase/deploy.sh with --prefetch-images-only and --skip-prefetch.

- Updated unit tests for CNPG backups and Supabase silent milestone sequencing.

- Included incidental environment updates from installer execution.

Co-authored-by: Junie <junie@jetbrains.com>
2026-03-02 00:34:01 -08:00