Commit Graph

34 Commits

Author SHA1 Message Date
chrisfu
c0c7c5539c Refactor namespace handling to use prole.cfg as sole source of truth
- Removed all logic that read or set NAMESPACE/PROLE_NAMESPACE from the shell environment or wrote it to env.sh.

- prole.cfg is now the sole source of truth for the namespace value, loaded exclusively by prole_cfg.sh.

- etc/prole_cfg.sh & mock_val/prole_cfg.sh: Removed kubectl-context fallback and default for PROLE_NAMESPACE.

- Shell scripts (init_*.sh): Replaced NAMESPACE=${NAMESPACE:-...} with NAMESPACE="${PROLE_NAMESPACE}".

- Python (actions, environment, milestone): Removed env["PROLE_NAMESPACE"] from subprocess env dicts and env.sh.

- etc/init_cloudnative_pg.sh: Removed DB_PASSWORD env fallback for secret creation.

Co-authored-by: Junie <junie@jetbrains.com>
2026-03-02 18:50:53 -08:00
chrisfu
6dc3560dcf refactor: rename backup script to etc/init_cnpg_backup.sh and optimize Supabase image prefetch
- Renamed etc/init_prole-db-backup.sh to etc/init_cnpg_backup.sh and updated all references.

- Enhanced CNPG backup script with authoritative status validation.

- Implemented SupabaseImagePreloadMilestone to front-load heavy image transfers.

- Updated supabase/deploy.sh with --prefetch-images-only and --skip-prefetch.

- Updated unit tests for CNPG backups and Supabase silent milestone sequencing.

- Included incidental environment updates from installer execution.

Co-authored-by: Junie <junie@jetbrains.com>
2026-03-02 00:34:01 -08:00
chrisfu
5618b662dd Remove prole-db-manager; simplify deployment via prole-authority; fix pg18 downgrade & cluster name
Summary:
Removed the prole-db-manager microservice and simplified deployment to use
prole-authority as the internal management and authorization point. Fixed two
blocking bugs that prevented silent install from completing on knoe-dev-cluster.

Removed: prole-db-manager
- Deleted db-manager-deployment.yaml and db-manager-service.yaml from opentofu manifests
- Deleted src/db-manager/ (Dockerfile, server.js, package.json, tests)
- Removed prole-db-manager port-forward mapping from installer/core/env.py
- Removed init_db_manager.sh from Initialization Scripts (milestones.py, actions.py)
- Removed init_certmgr.sh and init_db_manager.sh tabs from services screen (services.py)
- Removed live k8s Deployment/Service from knoe-dev-cluster

Fixed: PostgreSQL version downgrade error (pg17 -> pg18)
- Created conf/postgresql/.version with value 18
- Updated k8s/prole/prole-db.yaml and prole-db-recovery.yaml.tpl imageName to prole-db:18-089
- Fixed _init_database_options_state() to restore saved version_type from prole.cfg
  so db_version_type defaults to v18 (pg18) instead of silently reverting to pg17
- Added database_options.* keys to _collect_input_snapshot() in cfg.py so
  distribution, version_type, and all extension toggles persist to prole.cfg

Fixed: Cluster name inconsistency
- Removed stale prole-dev-cluster references; all scripts now use knoe-dev-cluster
- Added knoe-dev-cluster to mode-detection case in etc/prole_cfg.sh

Config: conf/prole.cfg
- Set kerberos_config.enabled = False, KERBEROS_AUTO_ENABLED = False
- Added database_options.distribution = percona, version_type = v18
- Added all 13 extension flags set to True (postgis, pgvector, pgcrypto, pgaudit,
  pg_repack, pg_stat_statements, pg_buffercache, pg_freespacemap, pgrowlocks,
  postgres_fdw, dblink, pg_stat_monitor, pgbadger)

Verification:
./install.py -s -l -v -c conf/prole.cfg completed successfully.
CNPG deployed prole-db:18-089 to knoe-dev-cluster; all milestones passed.

Co-authored-by: Junie <junie@jetbrains.com>
2026-03-01 20:40:44 -08:00
chrisfu
46f31843c2 Fix Kerberos test flow and improve multi-namespace support in k3s
- Resolve namespace drift in Kerberos test flow and OpenBao deployment.
- Fix OpenAPI credential-validation failures in etc scripts by using --validate=false with kubectl.
- Introduce preferred_service_namespace() to accurately target namespaces for KDC policy/test paths.
- Default Kerberos test user to 'administrator' and prefill from Ansible vault secrets.
- Generalized OpenBao deployment logic to correctly handle multi-namespace environments.
- Added regression test 'tests/etc/test_init_kerberos_k3s_real_kdc.sh'.
- Synchronized SERVICE_NAMESPACE extraction in prole_cfg.sh.

Co-authored-by: Junie <junie@jetbrains.com>
2026-03-01 00:57:10 -08:00
chrisfu
28b3cd72df fix(db-build): ensure Build Database Image uses valid Percona packages and non-interactive tzdata
- generator: map pg_stat_monitor to percona-pg-stat-monitor{major} for Percona 17/18 (was percona-postgresql-*-stat-monitor).

- templates: add non-interactive tzdata setup pinned to Etc/GMT-0 to avoid stdin prompts.

- templates: switch contrib to percona-postgresql-contrib metapackage.

- dockerfile: regenerated to reflect fixes; confirmed build succeeds.

Co-authored-by: Junie <junie@jetbrains.com>
2026-02-28 23:27:38 -08:00
chrisfu
f20df131df Fix Percona PG 18 build and update installer configuration.
- Dockerfile: Resolved interactive tzdata prompts with debconf and fixed Percona package names (percona-pg-stat-monitor18 and percona-postgresql-contrib).

- Configuration: Updated prole.cfg with specific namespace (prole-hq0-db0), user (chrisfu), and encrypted secrets.

- Installer: Enhanced initialization scripts (init_openbao.sh, init_certmgr.sh) and updated service layers for cluster deployment.

- Tests: Added coverage for dependent image collection in installer core.

- Version: Bumped prole-db version to 77.

Co-authored-by: Junie <junie@jetbrains.com>
2026-02-28 23:11:32 -08:00
chrisfu
5e0a1bda85 feat: Add GitOps (Gitea) and Supabase integration, plus database options
- Makefile: Added 'init' and 'deploy' targets for k3s parity and Gitea staging.

- OpenTofu: Fixed namespace handling in k3s main.tf to prevent metadata overwrites.

- UI: Added 'GitOps' and 'Database Options' configuration screens.

- Core: Enhanced monitoring, milestones, and environment handling for new services.

- Supabase: Integrated full Helm chart and manifest rendering logic.

- Gitea: Added deployment scripts and GitOps sync support.

- Database: Added Percona/Postgres Dockerfile templates and improved TDE scripts.

- Tests: Added coverage for new UI screens and navigation flows.
2026-02-26 18:32:15 -08:00
chrisfu
f2c9012cce Refactor installation and initialization logic, and expand test coverage
- install.py: Major update including configuration variable expansion, improved k3s/k3d handling, and enhanced installation logic.

- etc/ scripts: Significant refactoring of initialization scripts (Kerberos, Port Forwards, Garage Store, etc.).

- Port Forwards: Transitioned from XML to port-mappings.conf for managing kubectl port-forwards.

- Status Reporting: Improved status checking for common services.

- Infrastructure: Updated Ansible inventory and rsyslog role configurations.

- Tests: Added a comprehensive suite of tests for 'etc' initialization scripts in prole/tests/etc/.

- Documentation: Added prole-db-documentation-mcp-architecture.md.

- General: Updated Dockerfiles and various helper scripts.
2026-02-13 21:36:39 -08:00
chrisfu
96f594fd3c Refactor initialization scripts and add new service components
- Consolidated and split initialization scripts in etc/:
    - Removed init_prole-db.sh and init_authority.sh.
    - Added init_kdc.sh for in-cluster MIT Kerberos KDC (prole-authority).
    - Added init_ollama.sh for Ollama AI service integration.
    - Added init_service_layer.sh for high-level service orchestration.
    - Added init_k3s_registry.sh for private registry management.
- Major updates to install.py:
    - Support for new Ollama and KDC configuration.
    - Improved prole.cfg rendering and namespace handling.
    - Updated unattended install flags.
- Infrastructure and Deployment:
    - Updated K3s Ansible role with private registry support (registries.yaml template).
    - Added prole-authority Dockerfile.
    - Updated OpenBao Kerberos ConfigMap and other K8s manifests.
- Configuration:
    - Updated prole.cfg with new sections for Ollama and Monitoring.
    - Refined environment variable exports in env.sh and prole_cfg.sh.
2026-02-11 13:09:31 -08:00
chrisfu
da2f6600ba feat: infrastructure and installer updates for k3s, OpenTofu, and prole-db
- Add k3s start/stop Ansible playbooks and roles.

- Implement OpenTofu initialization scripts and k8s manifests.

- Update ncurses installer with OpenTofu support and improved k3s integration.

- Add mode support (--mode) to etc/ initialization scripts.

- Update prole-db with recovery, barman objectstore, and SSH OpenBao support.

- Refine k8s manifests for OpenBao and prole-db.
2026-02-05 21:27:18 -08:00
chrisfu
bcc8f23a0d Enhance secret management and k8s infrastructure
- Secret Management: Integrated AESGCM for temporary secret handling in install.py and enhanced OpenBao (Vault) support with namespace injection and additional secret paths (Grafana, Kerberos, TDE).
- Infrastructure & K8s:
    - Added Barman Object Store backup configuration (S3) to prole-db.yaml.
    - Updated Prometheus deployment with PVC and persistent configuration.
    - Updated k3s cluster/registry creation scripts.
    - Added etc/build-a-bao.sh for OpenBao setup.
- MSSQL Integration: Updated docker scripts and k8s deployments for Prole MSSQL database.
- Documentation: Added docs/PROLE-CFG-SECRETS.md explaining the new secret handling.
- General: Refined initialization scripts (init_authority.sh, init_openbao.sh, etc.) and updated the ncurses installer.
2026-02-03 22:39:50 -08:00
chrisfu
96aaf488d6 Bump Prole-DB image version to 17.7-059, enable at-rest encryption, and comment out Kerberos and pg_prolelog configurations. Update kdc and admin_server IPs in Kerberos config.
'Add CNPG build/init test and skip tiger geocoder' -m 'Test script usage: scripts/test-cnpg-prole-db.sh' -m 'Env overrides: CNPG_TEST_NAMESPACE, CNPG_CLUSTER_NAME, CNPG_MANIFEST_URL, CNPG_VERSION, IMAGE_LOAD (kind|minikube|k3d), IMAGE_LOAD_CMD, CNPG_INSTANCES'
2026-02-02 01:18:22 -08:00
chrisfu
7e9e6d0c11 Add CNPG build/init test and skip tiger geocoder
Test script usage: scripts/test-cnpg-prole-db.sh

Env overrides: CNPG_TEST_NAMESPACE, CNPG_CLUSTER_NAME, CNPG_MANIFEST_URL, CNPG_VERSION, IMAGE_LOAD (kind|minikube|k3d), IMAGE_LOAD_CMD, CNPG_INSTANCES
2026-02-02 01:15:11 -08:00
chrisfu
fff18fdbe4 Update Prole-DB and improve Supabase integration
- Bumped Prole-DB image version to 17.7-053 in scripts, Dockerfile, and manifests.
- Replaced `prole-scan` with `prole-agent` throughout scripts and tests.
- Refined Kubernetes setup for Supabase to use namespace 'supabase'.
- Introduced conversion of Supabase Docker Compose to Kubernetes manifests with `kompose`.
- Added support for Kerberos toggle via environment variables in `init_kerberos.sh`.
- Improved error handling and logging in scripts for better maintainability.
2026-02-01 23:56:25 -08:00
chrisfu
9c1f58caf4 Checkpoint: namespace management improvements and prole-db updates. Enhanced namespace management with automated namespace detection and configuration in etc/prole_cfg.sh and etc/init_prole-db.sh. Added namespace reset and recovery scripts: scripts/reset-ns.sh, etc/init_prole-db-reset.sh, and k8s/prole/prole-db-recovery.yaml.tpl. Updated prole-db image versioning to use release files and bumped version to 17.7-041. Improved installer UI and configuration handling in install.py and installer/screen.py. Updated Kerberos and CloudNativePG initialization scripts for better namespace support. Added /prole/backup/ to .gitignore. 2026-01-31 19:22:05 -08:00
chrisfu
dda938ee95 Integrate namespace setup and management into scripts, Dockerfile, and update installer UI. 2026-01-28 23:02:50 -08:00
chrisfu
5e823f82a8 feat(supabase): add optional init flow, k8s manifests, and docs
• install.py
• etc/init_supabase.sh
• k8s/prole/kustomization.yaml
• k8s/prole/prole-db-postgres-service.yaml
• k8s/prole/supabase-configmap.yaml
• k8s/prole/supabase-deployment.yaml
• k8s/prole/supabase-service.yaml
• prole-db/supabase.md
2026-01-27 13:06:35 -08:00
chrisfu
197c72dd7a Refactor prole-app and establish temporary release process
- Moved prole-tools-app to prole-app at the project root to make it self-contained for transition to its own repository.
- Created prole-tools-app/dist/ directory to host build artifacts.
- Generated distribution artifacts (Prole Tools.app and Prole Tools.zip) using prole-app/build.sh package.
- Checked in the generated artifacts to prole-tools-app/dist/ (bypassing .gitignore for temporary release process).
Changes Summary
•
Renamed directory prole-tools-app/ to prole-app/.
•
Populated prole-tools-app/dist/ with the latest build output from prole-app/build.sh.
•
Staged all changes, including the forced addition of ignored artifacts in prole-tools-app/dist/.
2026-01-18 15:04:23 -08:00
chrisfu
a4d0a2e0a3 Update prole-db docker tag to 17.7-037 and update pg_prolelog deb package 2026-01-15 11:58:39 -08:00
chrisfu
8df19ba986 updated pg_prolelog packages 2026-01-12 23:00:46 -08:00
chrisfu
bc2d051dc7 temporarily add pg_prolelog artifact 2026-01-12 12:01:41 -08:00
chrisfu
906392d462 feat(installer): improve UI and add test coverage for core features
- Refactored installer UI with updated canvas rendering, sidebar navigation, and footer buttons.
- Enhanced styling for macOS compatibility and consistent design across controls.
- Added Pytest-based unit tests for `screen.py` and `config.py`.
- Expanded dependency catalog with new tools like `tshark` and `pyshark`.
- Improved error tolerance for background rendering and added placeholders for Kerberos configuration.
2026-01-08 21:51:30 -08:00
chrisfu
b735a1df13 feat(app): add support for Ollama service integration
- Added Ollama endpoint to `Config.swift` and `install.py`.
- Updated `ServiceChecker` to handle Ollama status checks and errors.
- Enhanced `StatusView` with a sixth traffic light for Ollama.
2026-01-02 09:11:56 -08:00
chrisfu
73bc5bfede feat: implement encrypted development database with OpenBao and CNPG - Update install.py with password matching visual feedback and OpenBao integration - Enhance etc/init_openbao.sh to store database password in OpenBao KV - Update etc/init_cloudnative_pg.sh to synchronize database password from OpenBao to K8s secrets - Configure CNPG cluster in k8s/prole/prole-db.yaml with pg_hba for password authentication - Update various scripts and app sources for better service integration 2025-12-30 22:18:55 -08:00
chrisfu
940a48a237 refactor(scripts, installer): replace deprecated init_primary_domain.sh with init_openbao.sh
- Removed `init_primary_domain.sh` entirely; introduced `init_openbao.sh` with updated logic and streamlined functionality.
- Standardized script names to align with the OpenBao-centric workflow.
- Updated installer to dynamically fetch Prole database versions and reflect script changes.
- Improved TLS generation and k8s manifest handling for CloudNative-PG.
- Added new references to OpenBao initialization across scripts and UI components.
2025-12-23 22:40:03 -08:00
chrisfu
3f0f4b4c22 install.py drag and drop deploy. mv gcp to deploy dir. add init-port-fowards.sh and port-mappings.properties; now configured by Prole.app 2025-12-14 23:18:54 -08:00
chrisfu
603ce33dbf Updated prole-db/Dockerfile now based on percona-postgresql and uses pg_tde extension. openbao connections 2025-12-12 20:36:22 -08:00
chrisfu
ba2a957f4c Generate an animated loading GIF for Prole installer.
Inspired by 1980s Electronic Arts floppy disk loaders with alternating
animation speeds to reflect installation progress.
2025-12-12 19:56:03 -08:00
chrisfu
2020ca495c scram-sha-256 2025-11-10 12:58:15 -08:00
chrisfu
34b51d0d9b header 2025-11-06 10:47:00 -08:00
chrisfu
693a9434ed use k3s-arg datastore-endpoint external mysql 2025-08-29 13:03:02 -07:00
chrisfu
ac78325c7a very old prole gif 2025-08-13 15:46:30 -07:00
chrisfu
25224ea93b prole-db cloudnative-pg update 2025-07-29 14:04:10 -07:00
Chris Kifuthu
6e4f2cb4e7 cloudnative-pg deploy using prole-db image 2025-07-12 18:05:04 -07:00