Commit Graph

14 Commits

Author SHA1 Message Date
chrisfu
51c91595b9 Refactor infrastructure, installer, and deployment configs: expanded CloudNativePG and PostgREST init scripts, updated ArgoCD/OpenTofu/K8s manifests (ingress, kustomization, postgrest, prole-db), added installer config helpers and revised cluster UI, updated prole.cfg and prole.sh, simplified Makefile, added env helper tests, removed stale spec/log files, added prole/ directory 2026-02-23 21:19:19 -08:00
chrisfu
d68925bbc5 Add TLS certificates to CNPG cluster and restore missing Optional Features config - Add certificates section (prole-db-tls, prole-db-ca) to prole-db.yaml and prole-db-recovery.yaml.tpl - Expand generate_tls_if_missing() to create both CA and server TLS secrets with proper SANs - Add detect_and_reprovision_unencrypted_cluster() for existing clusters needing encryption conversion - Propagate AT_REST_ENCRYPTION_ENABLED to shell scripts via _script_env_for_namespace() - Restore AT_REST_ENCRYPTION_ENABLED, SUPABASE_ENABLED, KERBEROS_ENABLED in prole.cfg - Update installer milestones, docker screens, port-mapping, init_kerberos, init_postgrest, prole.sh 2026-02-22 18:58:01 -08:00
chrisfu
b03efa8f69 Kong API gateway, docker-import preload, OpenTofu graceful fallback, milestone fix
Kong API Gateway (replacing prole nginx):
- Add etc/init_kong.sh provisioning script (DB-less mode, prole-db namespace)
- Add kong-deployment.yaml and kong-service.yaml manifests
- Rewire ingress rules (svc/git/api.prole.org) to prole-db-kong:8000
- Update kustomization.yaml to reference kong manifests

PostgREST & DB Manager in prole-db namespace:
- Add etc/init_postgrest.sh and etc/init_db_manager.sh scripts
- Add postgrest/db-manager deployment and service manifests
- Add src/db-manager/ Node.js REST endpoint for backup triggers
- Default NAMESPACE changed to prole-db in both scripts

Docker image pre-load from PROLE_DATA/docker-import:
- Add _preload_docker_images() to init_common_services.sh
- Scan for .tar files exported by final_deployment.sh
- Import via k3d image import (k3d) or ctr (k3s) before deployments
- Increase rollout timeouts to 300s (configurable via ROLLOUT_TIMEOUT) in init_openbao.sh, init_opentofu.sh, init_garage_store.sh, init_registry.sh

OpenTofu password resolution fix:
- Add Kubernetes secret fallback in resolve_admin_password()
- Change hard exit 1 to graceful return 1 with warning
- Wrap call in if-guard so set -e doesn't abort the script chain

Milestone fix (init scripts not running):
- Add init_kong.sh, init_postgrest.sh, init_db_manager.sh to InitializationScriptsMilestone.execute() script list and arg branches
- Previously only actions.py had these; milestones.py was missing them

Installer integration:
- Add Kong/PostgREST/DB Manager to silent installer _step_init_scripts
- Add corresponding tabs and execution blocks in UI services.py
2026-02-22 00:57:49 -08:00
chrisfu
4bbd3b2dd3 fix: UI runtime fixes, idempotent k3d KUBECONFIG, Ollama screen redesign, mode-conditional init scripts, Grafana restart, real-time docker build output, shell script fixes, test updates 2026-02-20 23:19:48 -08:00
chrisfu
322ed0cbfb fix: clean install bugs + coverage support for prole.sh start command - DatabaseCreationMilestone: detect unresolvable OpenBao secret refs, generate fresh password - NetworkScreenMixin: guard _scan_frames with getattr, add lazy accessor - port-mapping.cfg: resolve NAMESPACE refs in UI and silent paths, skip MagicMock entries - init_k8s.sh: add --api-port and --agents flags to k3d cluster create - prole.sh: extend coverage flag support to start command 2026-02-20 19:18:32 -08:00
chrisfu
ec50f82cc8 refactor: deduplicate installer business logic & split screens.py into package
Separation of concerns: merge silent/UI actions & modularize screens.

ProleInstallerBase (actions.py): Created shared base class with 37 deduplicated methods previously duplicated between ProleSilentInstaller and ProleInstaller. Namespace, environment, secret, deployment, port-forward, authority/repair, image, and logging helpers now defined once. Subclasses override _get_input() to bridge their data-access layers.

screens.py -> screens/ package (18 mixin modules): Split 10,234-line monolithic screens.py into focused mixin modules: base, navigation, welcome, dependencies, network, environment, database, cluster, services, security, ollama, supabase, docker, build, packaging, deploy, validate, cfg. __init__.py composes ProleInstaller from all mixins and re-exports has_display(), main() for full backward compatibility.

All 37 tests pass with no regressions.
2026-02-20 14:43:20 -08:00
chrisfu
9094d10117 Installer refactor: k3d cluster lifecycle, kubeconfig handling, UI polish & cleanup - Fix KUBECONFIG not generated for dev/k3d clusters by merging k3d kubeconfig in _script_env_for_namespace() and after cluster creation - Fix Add/Delete button macOS black focus ring with highlightbackground/highlightthickness styling - Refactor milestones, actions, controller, deploy pipeline, and env helpers - Simplify prole.cfg and port-mapping.cfg defaults - Improve archive_garage_backup.py and init_cloudnative_pg.sh - Remove unused img/LOADING_GIF_README.md and img/generate_loading_gif.py - Add qodana.yaml for static analysis configuration 2026-02-20 01:28:52 -08:00
chrisfu
6e2d3e9011 refactor: modernize installer and monitoring setup
- Monitoring: Migrated from manual Grafana/Prometheus manifests to kube-prometheus-stack based setup in etc/init_monitoring.sh. Removed old manifest files from deploy/ and k8s/.
- Installer Core: Refactored installer with new modules for actions, environment handling, and UI screens. Enhanced Milestone logic to support advanced configuration (ArgoCD, Registry namespaces, Kerberos flags, etc.).
- Service & Init Scripts: Updated multiple initialization scripts (init_*.sh) for better integration with OpenBao, Kerberos, and the new monitoring stack. Added new scripts for Nginx Ingress, Ollama parsing, and K3D route fixes.
- Infrastructure: Enhanced Samba AD DC Ansible role with realm derivation, provisioning guidance, and group management. Updated K3s role tasks.
- Configuration: Refined default settings in conf/ to align with the new deployment architecture.
- App & Tools: Updated prole-app Swift code and prole.sh for improved environment variable handling and installation flow.
2026-02-19 21:07:39 -08:00
chrisfu
ec2795f8a3 Checkpoint: Kerberos egress fixes, local registry provisioning for k3d/k3s, and installer GUI workflow enhancements. Key changes: AD forwarder port offsets, outbound NetworkPolicy, KDC dns_lookup disabling; k3d/k3s registry support with Docker network connect fix; GUI 'Deploy Services' button and manual pause; new test_service_layer_navigation.py. 2026-02-17 14:26:12 -08:00
chrisfu
ec8a4e98af k3s: Fix installation hangs, CA mismatches, and arm64 networking
- Implement proactive CA hash verification and automated repair for mismatches

- Ensure agents prioritize discovered server tokens over stale vault values

- Fix K3s service hangs with explicit stop and killall before reinstall

- Add Retropie/Pi networking fixes (WiFi power save, wlan0 priority)

- Pin pre-staged images to stable, architecture-aware versions (arm64)

- Remove obsolete init-port-forwards and prole.cfg sync tasks

- Update k8s manifests and installer core logic with new tests
2026-02-16 23:30:27 -08:00
chrisfu
9aa959f61d Milestone: Balanced iSCSI IO, updated prole.sh launcher with log/verbose/debug/coverage/silent options, and enhanced installer logging 2026-02-16 16:44:18 -08:00
chrisfu
23cf4ac33d Configure K3s for retropie and add Traefik Ingress. Updated K3s templates for node-ip/flannel-iface, configured retropie host vars, enabled Traefik, added Ingress resources for Grafana, Prole, Kong, and Supabase, and set Postgres to LoadBalancer. 2026-02-16 04:28:40 -08:00
chrisfu
cad49cc0a6 Summary of recent repairs and infrastructure updates
Repairs and improvements:
- iSCSI: Added cleanup tasks to remove stale mounts and fstab entries. Improved robustness of iSCSI target management and added 'iscsi_absent_mounts' support.
- K3s:
    - Updated service start logic to accept 'activating' state, preventing premature failure during slow startups.
    - Improved service stop logic to safely handle missing or not-found services.
    - Ensured 'prole-installer' ServiceAccount and ClusterRoleBinding exist for K8s administration.
    - Added leader election and etcd tuning arguments (forgiving leases) to config.yaml.j2.
    - Removed deprecated 'prole-port-forwards' systemd service.
- Installer & Scripts:
    - Updated legacy_tk.py to support K3s mode, secret resolution for passwords, and better environment management (including ~/.prole/env.sh for service mode).
    - Updated init_ansible.sh to support PROLE_VAULT_PASS_FILE and ANSIBLE_VAULT_PASSWORD_FILE.
    - Improved directory and kubeconfig path resolution in prole_cfg.sh to support fallback to ~/.prole.
    - Enhanced Grafana password resolution in init_monitoring.sh.
    - Added automatic application of iSCSI StorageClass and PersistentVolumes in init_openbao.sh.
- General: Switched conf/prole.cfg to k3s deployment mode and updated vault_k3s.yml token.

New Ansible Tasks and Playbooks:
- infrastructure/playbooks/iscsi_cleanup.yml: Automates logout and removal of stale iSCSI node records.
- infrastructure/playbooks/prole_logs_migrate.yml: Orchestrates /prole/logs migration to iSCSI storage.
- infrastructure/playbooks/tmp_bao_dir.yml: Ensures host-level storage directories for OpenBao.
- infrastructure/playbooks/tmp_mount.yml: Utility to verify and enforce host-level mounts.
- infrastructure/playbooks/k3s_sync.yml: Added tasks to start K3s after sync and update local kubeconfig on the controller.
- Added 'Unmount stale iSCSI mounts' and 'Remove stale iSCSI fstab entries' to the iscsi role.
- Added 'Ensure prole-installer service account exists' to the k3s role.
2026-02-15 17:51:57 -08:00
chrisfu
6b89ea23d7 Stabilize deliverable k3d pipeline and refactor installer components
- Finalized stable, repeatable reset logic for the k3d pipeline.

- Refactored installer into modular components: core, milestone, runner, and state.

- Introduced new UI abstractions with support for ncurses and Tkinter.

- Updated initialization scripts and configurations for CloudNativePG, Kerberos, OpenBao, and Monitoring.

- Improved pipeline repair and port-forwarding mechanisms.
2026-02-15 00:03:24 -08:00