Commit Graph

11 Commits

Author SHA1 Message Date
chrisfu
a1177d71bc Add GitOps provider choice screen and cluster UI improvements
- Add _render_gitops_choice_page() to GitOpsScreenMixin for explicit
  Gitea / ArgoCD / None provider selection with radio buttons
- Extend cluster screen with GitOps provider radio group and canvas layout
- Refactor services screen layout and navigation registration
- Register new gitops choice screen in screens __init__ / navigation
- Expand Kong init scripts (etc/ and mock_val/) with additional logic
- Update init_cnpg_backup.sh with minor fix
- Refresh conf/service and conf/prod prole.cfg generated configs
- Update conf/port-mapping.cfg port entries
- Update Supabase Helm values.yaml
- Bump modes/k3s/knoe-db/.version
- Update network scan description
- Add/expand tests: test_cluster_screen_layout, test_gitops_choice_screen

Co-authored-by: Junie <junie@jetbrains.com>
2026-03-24 22:12:25 -07:00
chrisfu
6f29deba99 Supabase/CNPG stability: pin Supabase to merlin, reduce CNPG to 2 instances, fix supabase_enabled
Changes address resource pressure (merlin OOM with 3 CNPG + Supabase + monitoring) and
the supabase_enabled=false override in silent mode that blocked Supabase deploy.

- conf/service/prole.cfg:
  - SUPABASE_PV_NODE/SUPABASE_PRIMARY_NODE = merlin.prole.org (d002 storage)
  - SUPABASE_STORAGE_CLASS = merlin-local-iscsi-d002
  - CNPG_ELIGIBLE_NODES: removed pi.prole.org (pi NotReady, d003 inaccessible)
- k8s/prole/knoe-db.yaml:
  - instances: 3 -> 2 (merlin + myrddin only, pi excluded)
  - maxSyncReplicas: 1 -> 0 (safe for 2-node cluster)
- supabase/deploy.sh:
  - default SUPABASE_PV_NODE to merlin.prole.org, base dir to /synology/d002/supabase
  - add idempotent ALTER ROLE password sync after CREATE ROLE IF NOT EXISTS
  - resolve live DB_PASSWORD from knoe-db-superuser secret before helm render
- supabase/helm/render_supabase.py:
  - resolve SUPABASE_STORAGE_CLASS from env/cfg, apply to storage/analytics PVCs
  - DATABASE_NAMESPACE resolution: check env + cfg Global section before discovery
  - DB_PASSWORD from env takes priority over cfg (live cluster secret wins)
  - default primary node = merlin.prole.org
- knoe/ui/screens/cluster.py: minor display update

Co-authored-by: Junie <junie@jetbrains.com>
2026-03-24 13:52:38 -07:00
chrisfu
ad36f4ee54 Stage 2: complete CNPG shell-to-Python cutover, remove init_cloudnative_pg.sh
All active runtime paths that previously shelled out to etc/init_cloudnative_pg.sh
now dispatch through Python. The shell script has been removed from git.

New files:
- knoe/core/ops/cloudnative_pg.py: canonical Python owner for CNPG lifecycle with
  public API (initialize, deploy, rollout, ensure_operator, pin_controller,
  install_barman_plugin) and internal helpers (_apply_manifest, _wait_cnpg_pods,
  _reconcile_instances, etc.)
- prole/tools/run_cnpg_coverage.py: coverage entry points for k3d/k3s modes,
  report sub-command, and check-shell scanner to confirm no live Python dispatch
  to the removed shell script

Modified files:
- knoe/core/actions.py: replace shell dispatch in _step_init_scripts,
  _step_cnpg_deploy, and repair pipeline with Python calls
- knoe/core/milestones.py: replace shell dispatch in InitializationScriptsMilestone
  and DeploymentMilestone
- knoe/ui/screens/services.py: replace shell dispatch for init-scripts step,
  deploy button, and rollout button
- prole/deployment.py: replace shell dispatch in _run_post_apply_scripts
- status.py: remove init_cloudnative_pg.sh from _STATUS_SCRIPTS list
- tests/installer/test_actions_helpers.py: mock Python functions, assert shell
  script is never dispatched
- tests/installer/test_milestones.py: same
- tests/installer/test_services_init_scripts.py: same

Removed:
- etc/init_cloudnative_pg.sh (git rm)

Verification:
- prole.tools.run_cnpg_coverage check-shell reports clean
- All 61 tests in the affected test files pass

Co-authored-by: Junie <junie@jetbrains.com>
2026-03-23 19:28:04 -07:00
chrisfu
2505497f0f Standardize namespace and cluster config flow
- Replace ambiguous NAMESPACE handling with SERVICE_NAMESPACE, DATABASE_NAMESPACE, and CLUSTER_NAME across config parsing/defaults, environment setup, actions, and UI bindings.

- Persist DATABASE_NAMESPACE and CLUSTER_NAME from Database Browser on Next and use them consistently in CNPG deployment/runtime logic.

- Update milestones/tests and include related config/tooling artifacts (cleanup_cnpg_storage tool, service config/version/network updates).

Co-authored-by: Junie <junie@jetbrains.com>
2026-03-23 16:52:40 -07:00
chrisfu
cf51335ced Fix kube context handling and monitoring eligibility
- detect local k3s node kubeconfig and skip kubectx/use-context mutation when already targeting local API\n- add configurable KUBE_CONTEXT_NAME resolution with compatibility fallbacks and switch only when required\n- update init scripts to use ensure_kube_context helper naming\n- broaden monitoring eligibility to discovered /synology/d### mounts so /synology/d004 qualifies\n- add focused kube-context and topology tests covering local/remote and read-only kubeconfig cases

Co-authored-by: Junie <junie@jetbrains.com>
2026-03-23 13:50:29 -07:00
chrisfu
0e806a9233 checkpoint: tighten CNPG k3s storage flow and policy updates
- normalize K3S/CNPG/OpenTofu env keys across installer and configs; - harden CNPG storage validation/provisioning and fail fast in init script; - update DB manifests/recovery template and port/network mapping data; - expand cluster node policy UI coverage and shell/python tests; - add k3s mode version marker files

Co-authored-by: Junie <junie@jetbrains.com>
2026-03-23 12:39:03 -07:00
chrisfu
a069989315 Rename prole-db to knoe-db, add knoe-auth as cluster-internal KDC
Itemized changes:

1. knoe-auth: New cluster-internal KDC and SSO gateway service
   - Created etc/init_knoe_auth.sh based on init_kdc.sh with knoe-auth naming
   - Namespace defaults to SERVICE_NAMESPACE (knoe-system)
   - ConfigMap: knoe-auth-kdc-config, Secret: knoe-auth-secrets
   - Legacy cleanup removes old auth/dog/authority deployments

2. Orchestration: knoe-auth initializes before CloudNativePG
   - Updated prole.sh to insert init_knoe_auth.sh as step 2 (before CNPG)
   - Renumbered all subsequent initialization steps

3. Kong routing: Updated init_kong.sh to route to knoe-auth in SERVICE_NAMESPACE

4. Comment/reference updates for knoe-auth
   - Updated init_common_services.sh, init_service_layer.sh, init_kerberos.sh

5. prole-db renamed to knoe-db across the entire codebase
   - Renamed prole-db/ directory to knoe-db/
   - Renamed all prole-db Kubernetes manifests (deploy/opentofu, k8s/)
   - Renamed scripts: docker-root-knoe-db.sh, docker-run-knoe-db.sh, test-cnpg-knoe-db.sh
   - Renamed etc/init_prole-db-reset.sh to etc/init_knoe-db-reset.sh
   - Renamed etc/prole-db-passwwd.sh to etc/knoe-db-passwwd.sh
   - Renamed mock_val counterparts accordingly
   - Renamed tests/etc/test_init_prole-db-reset.sh to test_init_knoe-db-reset.sh
   - Renamed docs/prole-db-documentation-mcp-architecture.md to knoe-db variant
   - Renamed modes/k3d/prole-db/ to modes/k3d/knoe-db/
   - Renamed prole-db.iml to knoe-db.iml

6. Configuration updates
   - Updated conf/dev, conf/prod, conf/test, conf/service prole.cfg files
   - Updated conf/port-mapping.cfg
   - Updated etc/prole_cfg.sh and mock_val/prole_cfg.sh
   - Updated service/prole.cfg

7. Kubernetes manifests and deploy configuration
   - Updated deploy/opentofu/k3s ArgoCD application YAMLs
   - Updated kong-configmap.yaml and kustomization.yaml
   - Updated k3s/kong-config.yml and prole-resources.yaml
   - Updated prole-mssql-db deployment YAMLs
   - Updated supabase helm render and deploy scripts

8. Infrastructure and GCP Terraform
   - Updated deploy/gcp/terraform: folders, groups, IAM, service-projects

9. Python/installer code updates
   - Updated knoe/core: actions, build_context, controller, env, milestones
   - Updated knoe/milestone.py
   - Updated knoe/ui/screens: cfg, database, database_options, deploy, docker,
     navigation, security, services, validate
   - Updated knoe.spec, status.py

10. Shell script updates
    - Updated etc/: build_db, init_cloudnative_pg, init_cnpg_backup,
      init_db_manager, init_forgejo, init_gitlab, init_monitoring, init_openbao,
      init_port_forwards, init_postgrest, init_supabase_ports, status
    - Updated mock_val/ counterparts for all above scripts
    - Updated prole-net/init-prole-dns.sh
    - Updated bin/prole-kpf.sh, gitea/deploy.sh, supabase/deploy.sh

11. Test updates
    - Updated tests/etc/: test_init_cloudnative_pg*, test_init_cnpg_backup*,
      test_init_kdc*, test_init_kerberos*, test_init_kong*, test_prole_cfg*
    - Updated tests/installer/: test_actions_helpers, test_cfg_save_kubecontext,
      test_controller, test_core_classes, test_milestones, test_milestones_extended,
      test_namespace_propagation
    - Updated tests/: test_database_options, test_navigation,
      test_render_supabase_hostname, test_docker_build_fix,
      test_all_prole_home_fixes, silent_install_test, final_test

12. Documentation updates
    - Updated docs/: DOCKER-BUILD-FIX, PROLE-CFG-SECRETS, PROLE-HOME-DIRECTORY,
      build-system, patent
    - Updated scan/network_description.txt
    - Updated pom.xml

13. Miscellaneous script updates
    - Updated root-level: _adopt_replica_pvcs, _fix_replica_merlin, _import_pi,
      _patch_cluster, _prebind_pvcs, _rebind_d002, _rebind_d002b, test_resolve
    - Updated scripts/generate_spec.py

Co-authored-by: Junie <junie@jetbrains.com>
2026-03-22 22:16:21 -07:00
chrisfu
9b9d6fdc88 Rename Prole storage objects to Synology and sync deployment updates
- Rename iSCSI storage class and PV/PVC selectors/labels from prole to synology across k8s and OpenTofu manifests\n- Update CNPG/OpenBao/Garage/monitoring init flows, render helpers, and mock scripts for synology-backed storage objects\n- Integrate related UI/core/service config/version updates and add supporting regression tests for CNPG storage/image behavior\n- Keep storage reconciliation tests aligned with current CNPG affinity output

Co-authored-by: Junie <junie@jetbrains.com>
2026-03-22 21:50:02 -07:00
chrisfu
0262bd4be2 Refactor to support mode-scoped DB build contexts and versioning
- Introduce mode-scoped directories under `~/.prole` for build contexts, runtime data, and versioning.
- Update DB image version generation and Dockerfile creation to respect deployment modes (e.g., k3d, k3s, k8s).
- Adjust UI, core actions, and scripts to avoid cross-mode conflicts and ensure correct version isolation.
- Add tests for mode-aware behaviors and update existing tests for new paths and markers.
- Bump PostgreSQL and image versions to `18` and `138`, respectively.
- Update Kubernetes manifests to align with new image and version configuration.
2026-03-22 05:47:50 -07:00
chrisfu
1353882260 Update PROLE_HOME resolution and enhance iSCSI PV handling
- Implement centralized `resolve_prole_home` utility for consistent environment-based `PROLE_HOME` resolution across modules
- Replace hardcoded home paths with `resolve_prole_home`
- Refactor PV management to support iSCSI mounts and node placement from Ansible manifests
- Improve Kubernetes manifest handling to dynamically apply namespaces per document
- Adjust `knoe-db` build context path and related tests
- Add utilities for detecting and applying Ansible-defined node labels and PVs
2026-03-22 02:43:52 -07:00
chrisfu
4ee2b259c9 Checkpoint: rename installer to knoe + harden db build context
- Add build-context helper to copy Docker context safely (ignore runtime data, keep symlinks)

- Update UI and core actions to use ~/.prole/build and shared copy helper

- Add/adjust tests and scripts; introduce knoe ops helpers and update manifests

Co-authored-by: Junie <junie@jetbrains.com>
2026-03-22 01:45:21 -07:00