All active runtime paths that previously shelled out to etc/init_cloudnative_pg.sh
now dispatch through Python. The shell script has been removed from git.
New files:
- knoe/core/ops/cloudnative_pg.py: canonical Python owner for CNPG lifecycle with
public API (initialize, deploy, rollout, ensure_operator, pin_controller,
install_barman_plugin) and internal helpers (_apply_manifest, _wait_cnpg_pods,
_reconcile_instances, etc.)
- prole/tools/run_cnpg_coverage.py: coverage entry points for k3d/k3s modes,
report sub-command, and check-shell scanner to confirm no live Python dispatch
to the removed shell script
Modified files:
- knoe/core/actions.py: replace shell dispatch in _step_init_scripts,
_step_cnpg_deploy, and repair pipeline with Python calls
- knoe/core/milestones.py: replace shell dispatch in InitializationScriptsMilestone
and DeploymentMilestone
- knoe/ui/screens/services.py: replace shell dispatch for init-scripts step,
deploy button, and rollout button
- prole/deployment.py: replace shell dispatch in _run_post_apply_scripts
- status.py: remove init_cloudnative_pg.sh from _STATUS_SCRIPTS list
- tests/installer/test_actions_helpers.py: mock Python functions, assert shell
script is never dispatched
- tests/installer/test_milestones.py: same
- tests/installer/test_services_init_scripts.py: same
Removed:
- etc/init_cloudnative_pg.sh (git rm)
Verification:
- prole.tools.run_cnpg_coverage check-shell reports clean
- All 61 tests in the affected test files pass
Co-authored-by: Junie <junie@jetbrains.com>
- Replace ambiguous NAMESPACE handling with SERVICE_NAMESPACE, DATABASE_NAMESPACE, and CLUSTER_NAME across config parsing/defaults, environment setup, actions, and UI bindings.
- Persist DATABASE_NAMESPACE and CLUSTER_NAME from Database Browser on Next and use them consistently in CNPG deployment/runtime logic.
- Update milestones/tests and include related config/tooling artifacts (cleanup_cnpg_storage tool, service config/version/network updates).
Co-authored-by: Junie <junie@jetbrains.com>
- detect local k3s node kubeconfig and skip kubectx/use-context mutation when already targeting local API\n- add configurable KUBE_CONTEXT_NAME resolution with compatibility fallbacks and switch only when required\n- update init scripts to use ensure_kube_context helper naming\n- broaden monitoring eligibility to discovered /synology/d### mounts so /synology/d004 qualifies\n- add focused kube-context and topology tests covering local/remote and read-only kubeconfig cases
Co-authored-by: Junie <junie@jetbrains.com>
- Introduced `test_init_cloudnative_pg_manifest_validation_fail_fast.sh` script.
- Ensures runtime validation and pod readiness wait are skipped if manifest validation fails.
- Added comprehensive mocks for tools (e.g., `kubectl`, `findmnt`) to simulate behavior.
- Updated script to include detailed error and debug logs during manifest validation steps.
- Adjusted function logic to handle manifest validation failures gracefully without proceeding further.
- set global MODE/DEPLOYMENT_MODE to k3s and remove cross-environment/prod cluster entries from active dev/service config paths\n- switch k3s prole-auth image references to myrddin.prole.org:5000\n- strengthen CNPG image resolution and test coverage for rewriting k3d/localhost registries in k3s mode\n- include updated port mapping/network snapshot and current k3d mode version metadata
Co-authored-by: Junie <junie@jetbrains.com>
Add storage ops for deterministic namespace+cluster local PV paths, labels/selectors, host-path preparation, and idempotent reconciliation before CNPG cluster apply.
Wire selector injection and validation into CNPG deploy flow/script, and extend installer tests for provisioning orchestration and failure handling.
Co-authored-by: Junie <junie@jetbrains.com>
- Introduce `cnpg_placement.py` to handle round-robin node assignments for CloudNativePG clusters.
- Implement persistent placement plans with schema normalization and hashing.
- Add `load_cnpg_placement_plan` and `save_cnpg_placement_plan` for plan persistence.
- Integrate planner with installer to support node-based topology configuration.
- Update `actions.py` with placement planning logic, including rebalance support and node eligibility checks.
- Extend shell utilities (`init_cloudnative_pg.sh`) for placement-aware CNPG topology adjustments.
- Add comprehensive unit tests and integration tests for planner functionality, persistence, and shell environment exports.
- Integrate `init_registry.sh` into service-layer initialization and cleanup flows.
- Update dependency order: Registry initializes first to support downstream services.
- Refactor cleanup sequence: Registry stops last in reverse dependency order.
- Enhance action mapping for registry lifecycle operations.
- Establish deployment sequence: Registry → OpenBao → Garage → OpenTofu.
- Relocate `init_registry.sh` call earlier in flow to align with dependencies.
- Update warning messages for missing script scenarios.
- Reorganize `init_opentofu.sh` to execute last as it depends on prior services.
- Introduce dynamic endpoint candidate selection for registry pushes, prioritizing localhost:5000 when applicable.
- Refactor to remove reliance on port-forwarding; fallback to direct node import for k3s.
- Enhance registry endpoint probing using both HTTP and HTTPS.
- Consolidate docker and skopeo push logic for robustness in multi-endpoint scenarios.
- Update network discovery to reflect restructured data.
- Use `scp` to transfer image tar files to remote nodes before importing, ensuring reliability for large images.
- Clean up temporary files on remote nodes post-import or in case of failure.
- Extend readiness probe to support HTTP alongside HTTPS endpoints.
- _prole_cfg_bootstrap_env_layout now detects when conf_dir is already
an env-specific directory (dev, service, prod, test, k3d, k3s, k8s)
and skips creating env subdirectories to avoid conf/service/service/.
- When conf_dir is an env dir with a stale symlink, resolve it to a
regular file instead of creating nested paths.
- Remove spurious conf/service/service/prole.cfg from the repo.
- Replace conf/service/prole.cfg symlink with the actual config file.
Co-authored-by: Junie <junie@jetbrains.com>
- Rename iSCSI storage class and PV/PVC selectors/labels from prole to synology across k8s and OpenTofu manifests\n- Update CNPG/OpenBao/Garage/monitoring init flows, render helpers, and mock scripts for synology-backed storage objects\n- Integrate related UI/core/service config/version updates and add supporting regression tests for CNPG storage/image behavior\n- Keep storage reconciliation tests aligned with current CNPG affinity output
Co-authored-by: Junie <junie@jetbrains.com>
- Introduce mode-scoped directories under `~/.prole` for build contexts, runtime data, and versioning.
- Update DB image version generation and Dockerfile creation to respect deployment modes (e.g., k3d, k3s, k8s).
- Adjust UI, core actions, and scripts to avoid cross-mode conflicts and ensure correct version isolation.
- Add tests for mode-aware behaviors and update existing tests for new paths and markers.
- Bump PostgreSQL and image versions to `18` and `138`, respectively.
- Update Kubernetes manifests to align with new image and version configuration.
- Implement centralized `resolve_prole_home` utility for consistent environment-based `PROLE_HOME` resolution across modules
- Replace hardcoded home paths with `resolve_prole_home`
- Refactor PV management to support iSCSI mounts and node placement from Ansible manifests
- Improve Kubernetes manifest handling to dynamically apply namespaces per document
- Adjust `knoe-db` build context path and related tests
- Add utilities for detecting and applying Ansible-defined node labels and PVs
- Rename host mount namespace from /prole/d00x to /synology/d00x
- Make LUN ownership explicit per host in inventory (myrddin=d001; merlin=d002,d004; pi=d003)
- Add safe migration cleanup for legacy /prole/d00x mounts and stale /etc/fstab entries
- Update k3s/ArgoCD/OpenTofu manifests and PV node affinities to match new mount paths
- Improve iSCSI role check-mode behavior and add quiesce/detach flow in ansible.sh
- Add Cluster Nodes spreadsheet screen for host/service placement (Primary Host vs Enabled Here)
- Wire navigation to show Cluster Nodes after Cluster Environment (multi-node, non-k3d)
- Update screen registry/base helpers and related layout/navigation tests
- Refresh port mappings and config fixtures
- Persist and export supabase_hostname for canonical external Supabase entrypoint (db.prole.org)
- Render Supabase Helm ingress host and public URLs (API_EXTERNAL_URL/GOTRUE_SITE_URL/SUPABASE_PUBLIC_URL) from config
- Align tracked Traefik ingress manifests to db.prole.org
- Generalize service hostname/TLS wiring in k3s automation and refresh Kong/monitoring behavior
- Make optional workload policy checks deterministic when kube context is absent; add render tests
- Pin Prometheus/Alertmanager + Grafana to merlin.prole.org in k3s to avoid PV node-affinity issues
- Taint myrddin.prole.org as control-plane-only and add CNPG toleration
- Prevent init_cloudnative_pg from deploying prole-svc-kong into the DB namespace
- GitOps-manage Kong declarative config via ConfigMap + add placement/init tests
Prefer infra-managed manifests for bootstrap; reconcile CNPG instances based on Ready+schedulable labeled db nodes; update manifests to use node-role affinity + anti-affinity; add policy/tests and config touch-ups (incl. prole.cfg).
Update inventory/roles to keep K3s state on local storage and prevent iSCSI from managing /var/lib/rancher.
Also add a single-host k3s install playbook, a systemd override template, and docs describing the installer ↔ Ansible boundary and slow-storage knobs.
Co-authored-by: Junie <junie@jetbrains.com>
- Add diagnostics/reset/retry logic for stuck monitoring Helm installs\n- Add optional cluster-side node/lease deletion during k3s reset via delegated control-plane\n- Bump prole-db image tag/version to 18-123\n- Update monitoring init unit test for reset/diagnostics path
- k3s_sync: choose any server as sync source fallback; refresh kubeconfig and validate nodes\n- fetch_kubeconfig: save to prole-k3s.kubeconfig and etc/secrets/k3s.kubeconfig\n- k3s sync: sync server URL into config.yaml; restore TLS bundle only on servers\n- cert-manager/ACME: wait for CNI/webhook readiness; retry ClusterIssuer/Certificate apply\n- inventory: tune myrddin control-plane timing args; add render test\n- misc: update port mappings and prole-db manifests/version
Co-authored-by: Junie <junie@jetbrains.com>
- Materialize required DB Kubernetes secrets from user-entered password before CNPG init
- Auto-generate a temporary OpenTofu admin password in non-interactive runs when none is available
- Update installer UI flows and tests to cover the new preflights/rotation
Co-authored-by: Junie <junie@jetbrains.com>
- Introduce lib/shell helpers and keep etc/* scripts thin via compatibility shims
- Move Kerberos validation to scripts/validation/check_kerberos.sh and update callers
- Add deterministic shellspec unit tests under tests/shellspec/ and wire Maven to run them
- Add minimal Spring Boot authority module with startup + /health endpoint and Maven wiring
- Document the new layout in docs/layout.md
Co-authored-by: Junie <junie@jetbrains.com>
- Add `ansible.builtin.stat` task to verify `prole.cfg` availability before loading defaults.
- Prevent errors when `prole.cfg` is missing by skipping `lookup` operations.
- Update documentation to note `/external` may already be mounted before Ansible runs.
- Avoid redundant ownership and permissions enforcement in `mariadb_primary` role.
- Ensure dashboard service doesn't create an ordering cycle with `multi-user.target`.
- Verify service safely attaches to a virtual terminal (VT) and passes VT to Xorg.
- Ensure no `su` usage by default in the service.
- Validate script behavior for passing VT and `-keeptty` options to Xorg.
chore(dashboard): update systemd unit to avoid VT conflicts and improve logging
- Remove `After` and `Wants` directives for `multi-user.target` to prevent conflicts.
- Add `TTYPath`, `StandardInput`, and related options for VT attachment.
- Ensure `ExecStopPost` restores getty service for the dashboard VT.
fix(dashboard): improve Xorg resolution and clean up script logic
- Dynamically determine Xorg executable location in `dashboard.sh`.
- Pass VT, display, and `-keeptty` to Xorg to improve script reliability.
fix(mariadb): correct exFAT mount options for external storage
- Avoid `chown` failures by setting root ownership with group access for `mysql`.
- Update external mount options to `uid=0,gid=mysql,umask=0002`.
cleanup(mariadb): remove unnecessary owner/group permissions from migration marker task
- Enable mounting borrowed USB disk (exFAT) persistently at `/external`.
- Migrate MariaDB datadir to `/external/mariadb` with bind mount at `/srv/mariadb/mariadb`.
- Add migration marker and rollback support to prevent accidental remigration.
- Include Ansible tasks for external storage setup and migration validation.
- Add ssl_hosts group and prole_ssl role to deploy registry cert/key to /etc/ssl/certs/prole
- Configure k8s registry deployment to serve HTTPS using mounted host certs
- Switch k3s/containerd registry mirror config to https:// and prefer HTTPS checks with HTTP fallback
Co-authored-by: Junie <junie@jetbrains.com>
- Introduced `recycle_released_prole_iscsi_pv_for_pvc` to resolve stale claimRef issues during restarts.
- Updated `init_garage_store.sh` and `init_openbao.sh` to invoke the recycling logic when rollouts are stuck.
- Added regression tests to validate PV recycling for Garage and OpenBao services.
