mirror of
https://github.com/dredx/prole.git
synced 2026-09-23 10:13:58 +00:00
dba8a2d1dc
Junie's session targeted the prompt "stabilize ./install.py -c conf/k3d.cfg
using strict TDD" — broad installer-side work, not the k3d-mirror Phase 3
brief I had filed (which she didn't pick up; phase-3 brief stays open). All
750 installer tests pass post-change.
What Junie produced:
install.py (NEW) Top-level CLI entry point. Was
imagined by the prompt but didn't
exist; this commit makes it real.
knoe/deployment.py (NEW) `KnoeDeployment` orchestrator for
the k3s service-mode deploy pipeline.
Wraps Ansible kubeconfig fetch,
opentofu apply, init_*.sh post-apply
scripts, and (optionally) supabase/
deploy.sh.
knoe/ui/screens/cluster.py Dual-cluster GKE kubecontext UI: prod env
knoe/ui/screens/cfg.py now shows separate "App Cluster:" and
"DB Cluster:" dropdowns instead of a
single "Kubernetes Context:" combo.
New _app_kubectx_combo + _db_kubectx_combo
widgets; new app/db_cluster_kubecontext
tk.StringVars.
knoe/core/{actions,env,milestones}.py
knoe/core/ops/storage.py
knoe/config.py, knoe/knoe_conf.py Plumbing changes for the dual-cluster
kubecontext flow + storage-class topology
detection cleanup.
knoe/tools/cleanup_cnpg_storage.py (NEW) Stand-alone cleanup utility.
tools/dashboard.sh (NEW) Dashboard helper.
conf/knoe.cfg (NEW) Master cfg generated by knoe_conf.
conf/dev/ (NEW) Dev-mode cfg directory.
conf/port-mapping.cfg Port mapping tweaks for k3d.
tests/installer/* (8 files) New + extended tests for the dual-cluster
tests/test_database_options.py TUI, kubecontext save flow, storage ops,
topology detection, deploy helpers,
database-options screen.
Issues found in Junie's working state and fixed here:
1. install.py was a 11-line import shim with no shebang, no `chmod +x`,
no `if __name__ == '__main__'` block. `./install.py -c conf/k3d.cfg`
returned `Permission denied` and `python install.py` did nothing.
Added `#!/usr/bin/env python3`, `chmod +x`, and a __main__ block
that delegates to `knoe.ui.screens.main()`. `./install.py --help`
now prints the canonical argparse help.
2. knoe/deployment.py had FIVE `subprocess.run()` call sites with no
`timeout=` argument (`_run_script`, `_run_cmd`, the Ansible playbook
fetch, `tofu init`, `tofu apply`). A hung child process — typical
failure mode is a script waiting on stdin or a stalled network
call — would lock up the installer indefinitely. Added timeouts:
- Ansible kubeconfig fetch: 120s
- tofu init: 300s
- tofu apply, _run_script, _run_cmd: bounded by new module
constant `_MILESTONE_TIMEOUT` (default 1800s = 30 min, override
via `KNOE_MILESTONE_TIMEOUT_SECONDS` env var).
`subprocess.TimeoutExpired` is caught explicitly; on timeout the
run helpers return exit code 124 (conventional timeout code).
3. `conf/k3d.cfg` was corrupted with MagicMock string-reprs on disk:
KNOE_CONF = <MagicMock name='Canvas().tk.call().strip()' id='4743999712'>
argocd.node_selector = <MagicMock name='mock.StringVar().get().strip()' id='...'>
Likely path: Junie ran `./install.py -c conf/k3d.cfg` interactively
in a non-Tk environment (or with a partially-mocked widget set) and
the installer's "save current state" path wrote the mock-objects'
`__repr__` strings into the cfg file. This commit reverts the cfg
to its pre-Junie state. **Followup: harden the cfg save path
against non-string widget values** — track separately.
4. The corrupted cfg caused the installer to call `os.makedirs()` on
the mock-string values, producing 10 directories on disk literally
named `<MagicMock name='Canvas().tk.call().strip()' id='4733210304'>/`
etc., with 5–86 files of install artifacts inside each. Removed.
The "final step is timing out" the user reported was almost certainly
issue #2 above: install.py walked the milestone pipeline, hit one of
the unbounded subprocess.run calls, and the wrapped command (probably
supabase/deploy.sh, which Junie was reading for context when her
session timed out) hung. With the timeouts in place that path now
exits cleanly with rc=124 instead of locking up.
Verification:
- pytest tests/installer/ -q 750 passed in ~25s
- python3 -c "import knoe.deployment" imports clean
- ./install.py --help prints argparse help
- find . -maxdepth 1 -type d -name '<MagicMock*' | wc -l 0
- head -7 conf/k3d.cfg clean (no MagicMock)
Out of scope for this commit (followups):
- The cfg save-path that wrote mock-objects-as-strings (issue #3 root cause).
Reproducer: launch the installer in an env where Tk widget vars are
`unittest.mock.MagicMock` instances. The cfg save code should refuse to
serialize non-str values rather than calling `str()` on a MagicMock.
- The k3d-mirror Phase 3 brief (`docs/plans/junie/k3d-knoe-auth-pod-deploy.md`)
is still open — Junie picked a different prompt this round.
Co-authored-by: Junie <junie@jetbrains.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
191 lines
6.8 KiB
Python
191 lines
6.8 KiB
Python
from __future__ import annotations
|
|
|
|
import pytest
|
|
|
|
from knoe.core.ops import storage as storage_ops
|
|
|
|
|
|
def test_build_cluster_storage_paths_prefers_distinct_roots(monkeypatch):
|
|
monkeypatch.setattr(
|
|
storage_ops,
|
|
"_is_non_root_mount",
|
|
lambda path, **_kwargs: str(path) in {"/synology/d001", "/synology/d002"},
|
|
)
|
|
|
|
spec = storage_ops.ClusterStorageSpec(
|
|
namespace="knoe-db",
|
|
cluster_name="cluster-a",
|
|
node_name="myrddin.knoe.org",
|
|
synology_roots=("/synology/d001", "/synology/d002"),
|
|
)
|
|
|
|
paths = storage_ops.build_cluster_storage_paths(spec)
|
|
|
|
assert paths.data_root != paths.wal_root
|
|
assert paths.data_path.endswith("/knoe/knoe-db/knoe-db/cluster-a/data")
|
|
assert paths.wal_path.endswith("/knoe/knoe-db/knoe-db/cluster-a/wal")
|
|
|
|
|
|
def test_build_cluster_storage_paths_falls_back_to_same_root(monkeypatch):
|
|
monkeypatch.setattr(
|
|
storage_ops,
|
|
"_is_non_root_mount",
|
|
lambda path, **_kwargs: str(path) == "/synology/d001",
|
|
)
|
|
|
|
spec = storage_ops.ClusterStorageSpec(
|
|
namespace="knoe-db",
|
|
cluster_name="cluster-a",
|
|
node_name="myrddin.knoe.org",
|
|
synology_roots=("/synology/d001", "/synology/d002"),
|
|
)
|
|
|
|
paths = storage_ops.build_cluster_storage_paths(spec)
|
|
|
|
assert paths.data_root == "/synology/d001"
|
|
assert paths.wal_root == "/synology/d001"
|
|
assert paths.data_path != paths.wal_path
|
|
|
|
|
|
def test_build_pv_labels_include_cluster_scope():
|
|
labels = storage_ops.build_pv_labels(
|
|
namespace="knoe-db",
|
|
cluster_name="cluster-a",
|
|
role="data",
|
|
volume="d001",
|
|
)
|
|
|
|
assert labels["synology.storage/role"] == "data"
|
|
assert labels["synology.storage/volume"] == "d001"
|
|
assert labels["knoe.io/namespace"] == "knoe-db"
|
|
assert labels["knoe.io/cluster"] == "cluster-a"
|
|
assert labels["knoe.io/service"] == "knoe-db"
|
|
|
|
|
|
def test_provision_cluster_storage_fails_on_existing_name_mismatch(monkeypatch):
|
|
spec = storage_ops.ClusterStorageSpec(
|
|
namespace="knoe-db",
|
|
cluster_name="cluster-a",
|
|
node_name="myrddin.knoe.org",
|
|
)
|
|
paths = storage_ops.ClusterStoragePaths(
|
|
data_root="/synology/d001",
|
|
wal_root="/synology/d002",
|
|
data_volume="d001",
|
|
wal_volume="d002",
|
|
data_path="/synology/d001/knoe/knoe-db/knoe-db/cluster-a/data",
|
|
wal_path="/synology/d002/knoe/knoe-db/knoe-db/cluster-a/wal",
|
|
)
|
|
|
|
data_name = storage_ops.build_pv_name("synology-iscsi", "knoe-db", "cluster-a", "data")
|
|
monkeypatch.setattr(storage_ops, "build_cluster_storage_paths", lambda _spec: paths)
|
|
monkeypatch.setattr(storage_ops, "ensure_host_path", lambda *args, **kwargs: None)
|
|
|
|
existing = {
|
|
data_name: {
|
|
"metadata": {
|
|
"name": data_name,
|
|
"labels": storage_ops.build_pv_labels(
|
|
namespace="knoe-db",
|
|
cluster_name="cluster-a",
|
|
role="data",
|
|
volume="d001",
|
|
),
|
|
},
|
|
"spec": {
|
|
"storageClassName": "synology-iscsi",
|
|
"local": {"path": "/synology/d001/other-path"},
|
|
"nodeAffinity": {
|
|
"required": {
|
|
"nodeSelectorTerms": [
|
|
{
|
|
"matchExpressions": [
|
|
{
|
|
"key": "kubernetes.io/hostname",
|
|
"operator": "In",
|
|
"values": ["myrddin.knoe.org"],
|
|
}
|
|
]
|
|
}
|
|
]
|
|
}
|
|
},
|
|
},
|
|
}
|
|
}
|
|
monkeypatch.setattr(storage_ops, "_collect_existing_pvs", lambda _spec: existing)
|
|
|
|
with pytest.raises(storage_ops.StorageProvisioningError, match="unexpected path"):
|
|
storage_ops.provision_cluster_storage(spec)
|
|
|
|
|
|
def test_provision_cluster_storage_applies_and_returns_selectors(monkeypatch):
|
|
spec = storage_ops.ClusterStorageSpec(
|
|
namespace="knoe-db",
|
|
cluster_name="cluster-a",
|
|
node_name="myrddin.knoe.org",
|
|
)
|
|
paths = storage_ops.ClusterStoragePaths(
|
|
data_root="/synology/d001",
|
|
wal_root="/synology/d002",
|
|
data_volume="d001",
|
|
wal_volume="d002",
|
|
data_path="/synology/d001/knoe/knoe-db/knoe-db/cluster-a/data",
|
|
wal_path="/synology/d002/knoe/knoe-db/knoe-db/cluster-a/wal",
|
|
)
|
|
|
|
monkeypatch.setattr(storage_ops, "build_cluster_storage_paths", lambda _spec: paths)
|
|
monkeypatch.setattr(storage_ops, "_ensure_node_host_path", lambda *args, **kwargs: None)
|
|
monkeypatch.setattr(storage_ops, "ensure_host_path", lambda *args, **kwargs: None)
|
|
monkeypatch.setattr(storage_ops, "_collect_existing_pvs", lambda _spec: {})
|
|
|
|
applied: list[dict] = []
|
|
|
|
def _capture_apply(_spec, manifest):
|
|
applied.append(manifest)
|
|
|
|
monkeypatch.setattr(storage_ops, "_kubectl_apply_manifest", _capture_apply)
|
|
|
|
provisioned = storage_ops.provision_cluster_storage(spec)
|
|
|
|
assert len(applied) == 2
|
|
assert provisioned.data_pv_name == "synology-iscsi-knoe-db-cluster-a-data"
|
|
assert provisioned.wal_pv_name == "synology-iscsi-knoe-db-cluster-a-wal"
|
|
assert provisioned.data_selector["knoe.io/namespace"] == "knoe-db"
|
|
assert provisioned.data_selector["knoe.io/cluster"] == "cluster-a"
|
|
assert provisioned.wal_selector["synology.storage/role"] == "wal"
|
|
|
|
|
|
def test_provision_cluster_storage_fails_on_path_overlap(monkeypatch):
|
|
spec = storage_ops.ClusterStorageSpec(
|
|
namespace="knoe-db",
|
|
cluster_name="cluster-a",
|
|
node_name="myrddin.knoe.org",
|
|
)
|
|
paths = storage_ops.ClusterStoragePaths(
|
|
data_root="/synology/d001",
|
|
wal_root="/synology/d002",
|
|
data_volume="d001",
|
|
wal_volume="d002",
|
|
data_path="/synology/d001/knoe/knoe-db/knoe-db/cluster-a/data",
|
|
wal_path="/synology/d002/knoe/knoe-db/knoe-db/cluster-a/wal",
|
|
)
|
|
|
|
monkeypatch.setattr(storage_ops, "build_cluster_storage_paths", lambda _spec: paths)
|
|
monkeypatch.setattr(storage_ops, "ensure_host_path", lambda *args, **kwargs: None)
|
|
|
|
existing = {
|
|
"some-other-pv": {
|
|
"metadata": {"name": "some-other-pv", "labels": {}},
|
|
"spec": {
|
|
"storageClassName": "synology-iscsi",
|
|
"local": {"path": "/synology/d001/knoe/knoe-db/knoe-db/cluster-a"},
|
|
"nodeAffinity": {"required": {"nodeSelectorTerms": []}},
|
|
},
|
|
}
|
|
}
|
|
monkeypatch.setattr(storage_ops, "_collect_existing_pvs", lambda _spec: existing)
|
|
|
|
with pytest.raises(storage_ops.StorageProvisioningError, match="overlap"):
|
|
storage_ops.provision_cluster_storage(spec)
|