Commit Graph

9 Commits

Author SHA1 Message Date
chrisfu
dba8a2d1dc feat(installer): TDD stabilization for k3d install path; dual-cluster GKE TUI
Junie's session targeted the prompt "stabilize ./install.py -c conf/k3d.cfg
using strict TDD" — broad installer-side work, not the k3d-mirror Phase 3
brief I had filed (which she didn't pick up; phase-3 brief stays open). All
750 installer tests pass post-change.

What Junie produced:

  install.py                          (NEW) Top-level CLI entry point. Was
                                            imagined by the prompt but didn't
                                            exist; this commit makes it real.
  knoe/deployment.py                  (NEW) `KnoeDeployment` orchestrator for
                                            the k3s service-mode deploy pipeline.
                                            Wraps Ansible kubeconfig fetch,
                                            opentofu apply, init_*.sh post-apply
                                            scripts, and (optionally) supabase/
                                            deploy.sh.
  knoe/ui/screens/cluster.py          Dual-cluster GKE kubecontext UI: prod env
  knoe/ui/screens/cfg.py              now shows separate "App Cluster:" and
                                       "DB Cluster:" dropdowns instead of a
                                       single "Kubernetes Context:" combo.
                                       New _app_kubectx_combo + _db_kubectx_combo
                                       widgets; new app/db_cluster_kubecontext
                                       tk.StringVars.
  knoe/core/{actions,env,milestones}.py
  knoe/core/ops/storage.py
  knoe/config.py, knoe/knoe_conf.py   Plumbing changes for the dual-cluster
                                       kubecontext flow + storage-class topology
                                       detection cleanup.
  knoe/tools/cleanup_cnpg_storage.py  (NEW) Stand-alone cleanup utility.
  tools/dashboard.sh                  (NEW) Dashboard helper.
  conf/knoe.cfg                       (NEW) Master cfg generated by knoe_conf.
  conf/dev/                           (NEW) Dev-mode cfg directory.
  conf/port-mapping.cfg               Port mapping tweaks for k3d.
  tests/installer/* (8 files)         New + extended tests for the dual-cluster
  tests/test_database_options.py      TUI, kubecontext save flow, storage ops,
                                       topology detection, deploy helpers,
                                       database-options screen.

Issues found in Junie's working state and fixed here:

  1. install.py was a 11-line import shim with no shebang, no `chmod +x`,
     no `if __name__ == '__main__'` block. `./install.py -c conf/k3d.cfg`
     returned `Permission denied` and `python install.py` did nothing.
     Added `#!/usr/bin/env python3`, `chmod +x`, and a __main__ block
     that delegates to `knoe.ui.screens.main()`. `./install.py --help`
     now prints the canonical argparse help.

  2. knoe/deployment.py had FIVE `subprocess.run()` call sites with no
     `timeout=` argument (`_run_script`, `_run_cmd`, the Ansible playbook
     fetch, `tofu init`, `tofu apply`). A hung child process — typical
     failure mode is a script waiting on stdin or a stalled network
     call — would lock up the installer indefinitely. Added timeouts:
       - Ansible kubeconfig fetch: 120s
       - tofu init: 300s
       - tofu apply, _run_script, _run_cmd: bounded by new module
         constant `_MILESTONE_TIMEOUT` (default 1800s = 30 min, override
         via `KNOE_MILESTONE_TIMEOUT_SECONDS` env var).
     `subprocess.TimeoutExpired` is caught explicitly; on timeout the
     run helpers return exit code 124 (conventional timeout code).

  3. `conf/k3d.cfg` was corrupted with MagicMock string-reprs on disk:
        KNOE_CONF = <MagicMock name='Canvas().tk.call().strip()' id='4743999712'>
        argocd.node_selector = <MagicMock name='mock.StringVar().get().strip()' id='...'>
     Likely path: Junie ran `./install.py -c conf/k3d.cfg` interactively
     in a non-Tk environment (or with a partially-mocked widget set) and
     the installer's "save current state" path wrote the mock-objects'
     `__repr__` strings into the cfg file. This commit reverts the cfg
     to its pre-Junie state. **Followup: harden the cfg save path
     against non-string widget values** — track separately.

  4. The corrupted cfg caused the installer to call `os.makedirs()` on
     the mock-string values, producing 10 directories on disk literally
     named `<MagicMock name='Canvas().tk.call().strip()' id='4733210304'>/`
     etc., with 5–86 files of install artifacts inside each. Removed.

The "final step is timing out" the user reported was almost certainly
issue #2 above: install.py walked the milestone pipeline, hit one of
the unbounded subprocess.run calls, and the wrapped command (probably
supabase/deploy.sh, which Junie was reading for context when her
session timed out) hung. With the timeouts in place that path now
exits cleanly with rc=124 instead of locking up.

Verification:
  - pytest tests/installer/ -q                                   750 passed in ~25s
  - python3 -c "import knoe.deployment"                          imports clean
  - ./install.py --help                                          prints argparse help
  - find . -maxdepth 1 -type d -name '<MagicMock*' | wc -l       0
  - head -7 conf/k3d.cfg                                          clean (no MagicMock)

Out of scope for this commit (followups):
  - The cfg save-path that wrote mock-objects-as-strings (issue #3 root cause).
    Reproducer: launch the installer in an env where Tk widget vars are
    `unittest.mock.MagicMock` instances. The cfg save code should refuse to
    serialize non-str values rather than calling `str()` on a MagicMock.
  - The k3d-mirror Phase 3 brief (`docs/plans/junie/k3d-knoe-auth-pod-deploy.md`)
    is still open — Junie picked a different prompt this round.

Co-authored-by: Junie <junie@jetbrains.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-02 23:51:14 -07:00
chrisfu
55b6a6aff3 Complete rebranding from prole to knoe and fix macOS application identity. Bulk renamed 'prole' to 'knoe' across code, scripts, and manifests. Updated Makefile with 'knoe', 'build', and 'start' targets. Implemented macOS Application Bundle (.app) support for correct identity. Fixed macOS 'Python' process name to 'Knoe.DB Installer' via Objective-C bridge. Standardized application name to 'Knoe.DB Installer' across all interfaces.
Co-authored-by: Junie <junie@jetbrains.com>
2026-04-22 15:08:35 -07:00
chrisfu
fb7993de1d feat: add production config schema, validation, and API with tests
- Introduced `knoe.core.prod_config` with production configuration schema and helper methods.
- Added validation logic to enforce required fields and expected formats.
- Implemented in-process API (`ProdConfigApi`) for managing production configs.
- Added test coverage for default config behavior, validation, YAML generation, and API workflow.
- Updated cluster UI layout and tests to integrate production config with new navigation flows.
- Added host inventory and updated service flow to reflect production setup changes.
2026-03-28 10:43:25 -07:00
chrisfu
a1177d71bc Add GitOps provider choice screen and cluster UI improvements
- Add _render_gitops_choice_page() to GitOpsScreenMixin for explicit
  Gitea / ArgoCD / None provider selection with radio buttons
- Extend cluster screen with GitOps provider radio group and canvas layout
- Refactor services screen layout and navigation registration
- Register new gitops choice screen in screens __init__ / navigation
- Expand Kong init scripts (etc/ and mock_val/) with additional logic
- Update init_cnpg_backup.sh with minor fix
- Refresh conf/service and conf/prod prole.cfg generated configs
- Update conf/port-mapping.cfg port entries
- Update Supabase Helm values.yaml
- Bump modes/k3s/knoe-db/.version
- Update network scan description
- Add/expand tests: test_cluster_screen_layout, test_gitops_choice_screen

Co-authored-by: Junie <junie@jetbrains.com>
2026-03-24 22:12:25 -07:00
chrisfu
4ee2b259c9 Checkpoint: rename installer to knoe + harden db build context
- Add build-context helper to copy Docker context safely (ignore runtime data, keep symlinks)

- Update UI and core actions to use ~/.prole/build and shared copy helper

- Add/adjust tests and scripts; introduce knoe ops helpers and update manifests

Co-authored-by: Junie <junie@jetbrains.com>
2026-03-22 01:45:21 -07:00
chrisfu
0943186d08 installer: add Cluster Nodes screen + update flow
- Add Cluster Nodes spreadsheet screen for host/service placement (Primary Host vs Enabled Here)

- Wire navigation to show Cluster Nodes after Cluster Environment (multi-node, non-k3d)

- Update screen registry/base helpers and related layout/navigation tests

- Refresh port mappings and config fixtures
2026-03-20 16:15:47 -07:00
chrisfu
92e4101403 installer: preflight DB secrets + bootstrap OpenTofu password
- Materialize required DB Kubernetes secrets from user-entered password before CNPG init

- Auto-generate a temporary OpenTofu admin password in non-interactive runs when none is available

- Update installer UI flows and tests to cover the new preflights/rotation

Co-authored-by: Junie <junie@jetbrains.com>
2026-03-13 16:38:38 -07:00
chrisfu
d397b3000e Fix installer ctx apply flow and registry namespace
- Add explicit Apply button for kubectl context switching to avoid half-applied changes

- Allow kubeconfig/context-based auth without requiring K3S_TOKEN when kubeconfig is valid

- Prompt cleanup/reset of previous Common Core services namespace to prevent resource collisions

- Remove hardcoded 'common-services' registry namespace; default registry deploy/check to SERVICE_NAMESPACE/REGISTRY_NAMESPACE

- Update mocks and add regression tests for namespace resolution and installer behavior
2026-03-07 10:35:27 -08:00
chrisfu
765aa0b926 Checkpoint: cluster env UI layout + k3s common services
- Tighten Cluster Environment screen layout; switch Service/Prod to kubectx context selection; keep namespace and key controls on one line; ensure Repair button remains reachable.

- Add UI layout regression test to render with large mock data and assert key widgets remain visible and console is scrollable.

- Make kubeconfig generation deterministic under tests by avoiding overwriting cert-based kubeconfigs; write token sidecar kubeconfig when needed.

- Update common-services init scripts and add k3s/Helm deployment bits (svc-check, Kong/CertMgr tasks).
2026-02-27 14:35:14 -08:00