Commit Graph

13 Commits

Author SHA1 Message Date
chrisfu
2dc8726395 Skip Docker usage in init_kdc.sh when explicit KDC image is provided; add tests and improve kubectl retry handling 2026-03-08 22:52:45 -07:00
chrisfu
8134e27ec3 Add unit tests for init_cloudnative_pg and enable opt-in Docker pre-loading
- Introduced `test_init_cloudnative_pg_instances.sh` to validate CNPG instance count enforcement (default 3, configurable via `CNPG_INSTANCES`).
- Implemented support for opt-in Docker image pre-loading (`DOCKER_PRELOAD`/`PROLE_DOCKER_PRELOAD`) in `init_common_services.sh`.
- Added tests and validation for default pre-load behavior and toggle functionality in `test_init_common_services.sh`.
- Made `SERVICE_NAMESPACE` values consistent across configurations.
2026-03-08 21:53:49 -07:00
chrisfu
396cea60e7 installer: stabilize retries + improve k3s inventory detection
- Reuse namespace from existing prole.cfg unless env overrides

- Support k3s_hosts children groups and prefer k3s_servers as default

- Add headless pytest tkinter stubs and blocked-cluster reconciliation tests

- Update registry mirror endpoint, port mappings, and prole-db manifests
2026-03-07 23:57:21 -08:00
chrisfu
faca5d4a75 Add Python-native repair pipeline and tests; refactor bash wrapper
- Introduced `repair_pipeline_cli.py` to implement the repair pipeline in Python, enhancing maintainability and flexibility.
- Added `_repair_stale_released_pvs` method to handle reconciliation of stale Persistent Volumes and Persistent Volume Claims.
- Replaced complex bash-implemented repair logic in `repair_pipeline.sh` with a simplified wrapper that calls the Python script.
- Added unit tests for key repair methods, including `_match_stale_released_pv` and `_clear_pv_claim_ref`.
- Ensured support for namespace-scoped PVC repair logic and improved error handling during repair actions.
- Updated subprocess calls to handle interactions with `kubectl` within the pipeline's Python implementation.

This change simplifies cluster repair flows and improves future extensibility.
2026-03-07 14:42:06 -08:00
chrisfu
d397b3000e Fix installer ctx apply flow and registry namespace
- Add explicit Apply button for kubectl context switching to avoid half-applied changes

- Allow kubeconfig/context-based auth without requiring K3S_TOKEN when kubeconfig is valid

- Prompt cleanup/reset of previous Common Core services namespace to prevent resource collisions

- Remove hardcoded 'common-services' registry namespace; default registry deploy/check to SERVICE_NAMESPACE/REGISTRY_NAMESPACE

- Update mocks and add regression tests for namespace resolution and installer behavior
2026-03-07 10:35:27 -08:00
chrisfu
436c6214df checkpoint: k3s agent config + node labels + registry optimizations
- k3s: render server-only config keys only for servers; add regression test; make guardrail test non-sudo

- k3s/site: add post-provision node labeling tasks + playbook tier; extend inventory node label mapping

- registry: default namespace to common-services and skip redundant image pushes by probing registry manifests (installer + init scripts)
2026-03-07 03:14:26 -08:00
chrisfu
125da5669b Fix k3s CNPG image registry resolution and import flows
- In k3s mode, resolve CNPG images to a cluster-reachable internal registry and ignore k3d/localhost-ish registry values.

- Avoid any k3d cluster/containerd interactions when PROLE_MODE=k3s; add a preflight-image action.

- Extend init scripts and Ansible k3s/mariadb import tasks/playbooks for the updated k3s flow.

- Add/extend installer UI + tests around services and action/milestone helpers.

- Add merlin MariaDB provisioning playbook, services init-script test, and a k3s datastore SQL snapshot.

Co-authored-by: Junie <junie@jetbrains.com>
2026-03-07 00:23:00 -08:00
chrisfu
c0c7c5539c Refactor namespace handling to use prole.cfg as sole source of truth
- Removed all logic that read or set NAMESPACE/PROLE_NAMESPACE from the shell environment or wrote it to env.sh.

- prole.cfg is now the sole source of truth for the namespace value, loaded exclusively by prole_cfg.sh.

- etc/prole_cfg.sh & mock_val/prole_cfg.sh: Removed kubectl-context fallback and default for PROLE_NAMESPACE.

- Shell scripts (init_*.sh): Replaced NAMESPACE=${NAMESPACE:-...} with NAMESPACE="${PROLE_NAMESPACE}".

- Python (actions, environment, milestone): Removed env["PROLE_NAMESPACE"] from subprocess env dicts and env.sh.

- etc/init_cloudnative_pg.sh: Removed DB_PASSWORD env fallback for secret creation.

Co-authored-by: Junie <junie@jetbrains.com>
2026-03-02 18:50:53 -08:00
chrisfu
6dc3560dcf refactor: rename backup script to etc/init_cnpg_backup.sh and optimize Supabase image prefetch
- Renamed etc/init_prole-db-backup.sh to etc/init_cnpg_backup.sh and updated all references.

- Enhanced CNPG backup script with authoritative status validation.

- Implemented SupabaseImagePreloadMilestone to front-load heavy image transfers.

- Updated supabase/deploy.sh with --prefetch-images-only and --skip-prefetch.

- Updated unit tests for CNPG backups and Supabase silent milestone sequencing.

- Included incidental environment updates from installer execution.

Co-authored-by: Junie <junie@jetbrains.com>
2026-03-02 00:34:01 -08:00
chrisfu
5618b662dd Remove prole-db-manager; simplify deployment via prole-authority; fix pg18 downgrade & cluster name
Summary:
Removed the prole-db-manager microservice and simplified deployment to use
prole-authority as the internal management and authorization point. Fixed two
blocking bugs that prevented silent install from completing on knoe-dev-cluster.

Removed: prole-db-manager
- Deleted db-manager-deployment.yaml and db-manager-service.yaml from opentofu manifests
- Deleted src/db-manager/ (Dockerfile, server.js, package.json, tests)
- Removed prole-db-manager port-forward mapping from installer/core/env.py
- Removed init_db_manager.sh from Initialization Scripts (milestones.py, actions.py)
- Removed init_certmgr.sh and init_db_manager.sh tabs from services screen (services.py)
- Removed live k8s Deployment/Service from knoe-dev-cluster

Fixed: PostgreSQL version downgrade error (pg17 -> pg18)
- Created conf/postgresql/.version with value 18
- Updated k8s/prole/prole-db.yaml and prole-db-recovery.yaml.tpl imageName to prole-db:18-089
- Fixed _init_database_options_state() to restore saved version_type from prole.cfg
  so db_version_type defaults to v18 (pg18) instead of silently reverting to pg17
- Added database_options.* keys to _collect_input_snapshot() in cfg.py so
  distribution, version_type, and all extension toggles persist to prole.cfg

Fixed: Cluster name inconsistency
- Removed stale prole-dev-cluster references; all scripts now use knoe-dev-cluster
- Added knoe-dev-cluster to mode-detection case in etc/prole_cfg.sh

Config: conf/prole.cfg
- Set kerberos_config.enabled = False, KERBEROS_AUTO_ENABLED = False
- Added database_options.distribution = percona, version_type = v18
- Added all 13 extension flags set to True (postgis, pgvector, pgcrypto, pgaudit,
  pg_repack, pg_stat_statements, pg_buffercache, pg_freespacemap, pgrowlocks,
  postgres_fdw, dblink, pg_stat_monitor, pgbadger)

Verification:
./install.py -s -l -v -c conf/prole.cfg completed successfully.
CNPG deployed prole-db:18-089 to knoe-dev-cluster; all milestones passed.

Co-authored-by: Junie <junie@jetbrains.com>
2026-03-01 20:40:44 -08:00
chrisfu
46f31843c2 Fix Kerberos test flow and improve multi-namespace support in k3s
- Resolve namespace drift in Kerberos test flow and OpenBao deployment.
- Fix OpenAPI credential-validation failures in etc scripts by using --validate=false with kubectl.
- Introduce preferred_service_namespace() to accurately target namespaces for KDC policy/test paths.
- Default Kerberos test user to 'administrator' and prefill from Ansible vault secrets.
- Generalized OpenBao deployment logic to correctly handle multi-namespace environments.
- Added regression test 'tests/etc/test_init_kerberos_k3s_real_kdc.sh'.
- Synchronized SERVICE_NAMESPACE extraction in prole_cfg.sh.

Co-authored-by: Junie <junie@jetbrains.com>
2026-03-01 00:57:10 -08:00
chrisfu
5e0a1bda85 feat: Add GitOps (Gitea) and Supabase integration, plus database options
- Makefile: Added 'init' and 'deploy' targets for k3s parity and Gitea staging.

- OpenTofu: Fixed namespace handling in k3s main.tf to prevent metadata overwrites.

- UI: Added 'GitOps' and 'Database Options' configuration screens.

- Core: Enhanced monitoring, milestones, and environment handling for new services.

- Supabase: Integrated full Helm chart and manifest rendering logic.

- Gitea: Added deployment scripts and GitOps sync support.

- Database: Added Percona/Postgres Dockerfile templates and improved TDE scripts.

- Tests: Added coverage for new UI screens and navigation flows.
2026-02-26 18:32:15 -08:00
chrisfu
dafed53810 Enhance build system with 'make test' and fix installer test debris. Added 'make test' to Makefile with 'pyconv' (black) integration and coverage summary. Reformatted codebase with black. Fixed 'install.py' test bug where MagicMock objects created directory debris by improving mocking and patching in 'tests/test_navigation.py', 'tests/test_service_layer_navigation.py', and 'tests/test_install_logic.py'. 2026-02-25 14:55:05 -08:00