Replace all <MagicMock ...> values with empty strings or sensible defaults:
- BooleanVar mocks → false
- StringVar mocks for known fields (cluster_name, db_host_port, db_namespace,
db_username, cluster_env) → correct defaults
- All other StringVar mocks → empty string
Files are now safe to parse and commit. Protected going forward by
_validate_cfg_values() added in ef20c8a.
docs/completed/ — new directory; 9 shipped Junie briefs moved from
docs/plans/junie/ (02-k3s-prole-rename, 03-image-rename, 06-patch-garage,
07-init-cnpg-gke-sa-wiring, 13-podmonitor, 15-remove-dead-dashboard,
k3d-knoe-auth-dev-loop, phase2-oidc-gke-deploy, todo-1-cfg-save-path-bug).
docs/completed/README.md summarises all shipped work with dates/refs.
docs/plans/junie/README.md — updated to 2026-05-23; active/pending tables
reflect current state; shipped section now points to docs/completed/.
conf/service/knoe.cfg — new unattended deploy config for the service/k3s
environment (used by ./deploy.sh).
Co-authored-by: Junie <junie@jetbrains.com>
- knoe-db.yaml: switch to CNPG-managed TLS cert with serverAltDNSNames
(pg.prole.org + knoe-db-rw cluster service) — removes static serverTLSSecret/serverCASecret
- dns.yml: add pg.prole.org A record to prole_k3s_dns_records (10.0.0.3, 10.0.0.6)
for Ansible-managed split-horizon DNS via Samba AD DC
- k3s.cfg: align KNOE_HOME paths to ~/dev/prole, add PROLE_KDC_* vars, remove
hardcoded KUBECTL_CONTEXT (kubeconfig current-context is authoritative)
- prod.cfg: add PROLE_KDC_STORAGE_CLASS = prole-iscsi
- onepassword.py: skip vault check gracefully when no 1Password session active (non-TTY)
- CLAUDE.md: document production postgres connection string and DNS/CA cert ops
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Password resolution order for PROLE_KDC_MASTER_PASSWORD and KNOE_GUEST_PASSWORD:
1. Env var (existing)
2. knoe-kdc-secrets k8s Secret (existing)
3. prole-kdc-secrets k8s Secret (new — used by live prole deployment)
4. 1Password via op read (new — OP_KDC_MASTER_PASSWORD_REF / OP_KDC_GUEST_PASSWORD_REF)
5. PROLE_LOCAL_ADMIN_PASSWORD (existing)
6. die with actionable message
Add try_op_read() helper: wraps op CLI, returns 1 if op not available or ref empty.
conf/k3s.cfg: add KNOE_ADMIN_PRINCIPAL=chrisfu, OP_KDC_MASTER_PASSWORD_REF,
OP_KDC_GUEST_PASSWORD_REF placeholders (empty — user fills in op:// URIs).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
init_gitea.sh requires GITEA_PV_NODE in k3s mode. PVs for Gitea storage
(gitea-local-d005) are pinned to gandalf.prole.org per nodeAffinity on
the existing gitea-d005-shared-storage PV.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
k8s/knoe/iscsi-pvs.yaml had *.knoe.org nodeAffinity after the rename, but the
live k3s cluster nodes are still *.prole.org — causing immutable field errors on
Bound PVs during kubectl apply. Revert to *.prole.org to match live state.
conf/k3s.cfg KUBECTL_CONTEXT was renamed to knoe-service-cluster, but both the
system kubeconfig and etc/secrets/k3s.kubeconfig still use prole-service-cluster.
install.sh _check_gcp_tools() now skips gcloud check for k3s mode and any non-gke
config path (only GKE deploys need gcloud/kubectl pre-flight).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
conf/dev/knoe.cfg had a stale SERVICE_NAMESPACE=default written by a prior TUI session.
Also guard _service_namespace() so the value 'default' is never returned — it is never
a valid service namespace and signals a stale generated config.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
conf/service/prod.cfg: prole.org homelab overrides
- SERVICE_HOST_DOMAIN=prole.org, DB_UI_HOST=db.prole.org
- OIDC_ISSUER_URL=https://api.prole.org/auth
- GOOGLE_WORKSPACE_DOMAIN=prole.org (credentials in OpenBao bao/kv/knoe/oauth2/google-prole-org)
- kerberos_config: enabled, realm=PROLE.ORG, kdc=10.0.0.3 (myrddin.prole.org)
- PROLE_KDC_TRUST_REALM=PROLE.ORG, SERVICE_INGRESS_TLS_ENABLED=1
conf/k3d.cfg: update local dev paths to ~/dev/prole; enable kerberos for dev
env.sh: KNOE_HOME points to ~/dev/prole
application-prole.yml: Spring profile with prole.org Google Workspace client binding
These files are prole.org homelab specifics — do NOT merge upstream to knoe/main.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Junie's session targeted the prompt "stabilize ./install.py -c conf/k3d.cfg
using strict TDD" — broad installer-side work, not the k3d-mirror Phase 3
brief I had filed (which she didn't pick up; phase-3 brief stays open). All
750 installer tests pass post-change.
What Junie produced:
install.py (NEW) Top-level CLI entry point. Was
imagined by the prompt but didn't
exist; this commit makes it real.
knoe/deployment.py (NEW) `KnoeDeployment` orchestrator for
the k3s service-mode deploy pipeline.
Wraps Ansible kubeconfig fetch,
opentofu apply, init_*.sh post-apply
scripts, and (optionally) supabase/
deploy.sh.
knoe/ui/screens/cluster.py Dual-cluster GKE kubecontext UI: prod env
knoe/ui/screens/cfg.py now shows separate "App Cluster:" and
"DB Cluster:" dropdowns instead of a
single "Kubernetes Context:" combo.
New _app_kubectx_combo + _db_kubectx_combo
widgets; new app/db_cluster_kubecontext
tk.StringVars.
knoe/core/{actions,env,milestones}.py
knoe/core/ops/storage.py
knoe/config.py, knoe/knoe_conf.py Plumbing changes for the dual-cluster
kubecontext flow + storage-class topology
detection cleanup.
knoe/tools/cleanup_cnpg_storage.py (NEW) Stand-alone cleanup utility.
tools/dashboard.sh (NEW) Dashboard helper.
conf/knoe.cfg (NEW) Master cfg generated by knoe_conf.
conf/dev/ (NEW) Dev-mode cfg directory.
conf/port-mapping.cfg Port mapping tweaks for k3d.
tests/installer/* (8 files) New + extended tests for the dual-cluster
tests/test_database_options.py TUI, kubecontext save flow, storage ops,
topology detection, deploy helpers,
database-options screen.
Issues found in Junie's working state and fixed here:
1. install.py was a 11-line import shim with no shebang, no `chmod +x`,
no `if __name__ == '__main__'` block. `./install.py -c conf/k3d.cfg`
returned `Permission denied` and `python install.py` did nothing.
Added `#!/usr/bin/env python3`, `chmod +x`, and a __main__ block
that delegates to `knoe.ui.screens.main()`. `./install.py --help`
now prints the canonical argparse help.
2. knoe/deployment.py had FIVE `subprocess.run()` call sites with no
`timeout=` argument (`_run_script`, `_run_cmd`, the Ansible playbook
fetch, `tofu init`, `tofu apply`). A hung child process — typical
failure mode is a script waiting on stdin or a stalled network
call — would lock up the installer indefinitely. Added timeouts:
- Ansible kubeconfig fetch: 120s
- tofu init: 300s
- tofu apply, _run_script, _run_cmd: bounded by new module
constant `_MILESTONE_TIMEOUT` (default 1800s = 30 min, override
via `KNOE_MILESTONE_TIMEOUT_SECONDS` env var).
`subprocess.TimeoutExpired` is caught explicitly; on timeout the
run helpers return exit code 124 (conventional timeout code).
3. `conf/k3d.cfg` was corrupted with MagicMock string-reprs on disk:
KNOE_CONF = <MagicMock name='Canvas().tk.call().strip()' id='4743999712'>
argocd.node_selector = <MagicMock name='mock.StringVar().get().strip()' id='...'>
Likely path: Junie ran `./install.py -c conf/k3d.cfg` interactively
in a non-Tk environment (or with a partially-mocked widget set) and
the installer's "save current state" path wrote the mock-objects'
`__repr__` strings into the cfg file. This commit reverts the cfg
to its pre-Junie state. **Followup: harden the cfg save path
against non-string widget values** — track separately.
4. The corrupted cfg caused the installer to call `os.makedirs()` on
the mock-string values, producing 10 directories on disk literally
named `<MagicMock name='Canvas().tk.call().strip()' id='4733210304'>/`
etc., with 5–86 files of install artifacts inside each. Removed.
The "final step is timing out" the user reported was almost certainly
issue #2 above: install.py walked the milestone pipeline, hit one of
the unbounded subprocess.run calls, and the wrapped command (probably
supabase/deploy.sh, which Junie was reading for context when her
session timed out) hung. With the timeouts in place that path now
exits cleanly with rc=124 instead of locking up.
Verification:
- pytest tests/installer/ -q 750 passed in ~25s
- python3 -c "import knoe.deployment" imports clean
- ./install.py --help prints argparse help
- find . -maxdepth 1 -type d -name '<MagicMock*' | wc -l 0
- head -7 conf/k3d.cfg clean (no MagicMock)
Out of scope for this commit (followups):
- The cfg save-path that wrote mock-objects-as-strings (issue #3 root cause).
Reproducer: launch the installer in an env where Tk widget vars are
`unittest.mock.MagicMock` instances. The cfg save code should refuse to
serialize non-str values rather than calling `str()` on a MagicMock.
- The k3d-mirror Phase 3 brief (`docs/plans/junie/k3d-knoe-auth-pod-deploy.md`)
is still open — Junie picked a different prompt this round.
Co-authored-by: Junie <junie@jetbrains.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Implement Google OIDC support in Authority module via GoogleOAuthService
- Update AuthProperties and application.yml with OIDC configuration
- Add oidc-setup.md documentation for GKE/Google Cloud setup
- Update etc/init_knoe_auth.sh to handle OIDC secrets and path-B configuration
- Configure knoe-auth-deployment.yaml and gke.cfg for production auth
Co-authored-by: Junie <junie@jetbrains.com>
Discovered today after the SSH-ingress reconcile: init_gitlab.sh had a
destructive false-positive in its "legacy storage detection" at the
top of setup_gitlab_storage(). The bare check
if kubectl -n "$NAMESPACE" get pvc repo-data-gitlab-gitaly-0 >/dev/null 2>&1;
then has_legacy=1; fi
flagged has_legacy=1 whenever the PVC object simply existed — which is
always true after the first successful deploy. Combined with
GITLAB_REPAIR_BLOCKED_AUTOCLEAN=1, every init_gitlab.sh invocation
triggered the "repairing legacy Gitaly storage" branch, which scales
the Gitaly StatefulSet to 0 and deletes repo-data-gitlab-gitaly-0
immediately, wiping all git-repo data.
We caught it when the knoe-db repo (pushed via HTTPS earlier in the
day) disappeared after an unrelated init_gitlab.sh run for SSH ingress
wiring. The PVC age on inspection was ~26 min — newer than the push.
git ls-remote against the project now returns a garbled gzip body,
consistent with an empty gitaly backing an existing DB project.
Fix:
- etc/init_gitlab.sh:setup_gitlab_storage()
* Remove the bare `pvc exists` legacy check.
* Keep the real legacy markers: synology PV, gandalf.prole.org
nodeSelector, gitlab-gitaly-static storageClass on the STS.
* Add a new check that flags has_legacy=1 only when the PVC's actual
storageClassName is a known-legacy value (gitlab-gitaly-static or
synology-iscsi), never on mere existence.
- conf/gke.cfg
* GITLAB_REPAIR_BLOCKED_AUTOCLEAN: 1 -> 0. The script fix above makes
re-enabling safe, but the conservative default is to let the user
see repair_blocked and consent rather than silently wipe data.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Course-correction on 82a2c2e: git.knoe.dev was already fully green at
34.107.159.254 (ingress Active, managed cert Active, DNS pointed at
it). Annotating the frontdoor ingress with
kubernetes.io/ingress.global-static-ip-name: git-knoe-dev would force
the GKE LB controller to swap the forwarding rule to the new reserved
address, which means a fresh IP, a DNS flip, and ~5-15 min of managed
cert re-Provisioning -- for zero functional benefit.
Blank GITLAB_GLOBAL_STATIC_IP_NAME in conf/gke.cfg. The surface
(env var handling in etc/init_gitlab.sh and the annotation conditional
in the CR render) stays intact -- it's just opt-out. When the gitlab
ingress next needs to recreate for another reason, reserve
git-knoe-dev with that IP before the recreate and set this key.
The three other ingresses (svc-knoe-ingress, supabase-kong,
supabase-studio) stay pinned, because they were all already in a
broken state (wrong DNS and/or Provisioning cert) -- swapping costs
nothing there.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Reserved 5 named external static IPs and threaded the
kubernetes.io/ingress.global-static-ip-name annotation through every
public GCE L7 ingress the installer renders, so ingress delete/recreate
stops churning DNS. Also turned on a dedicated regional L4
LoadBalancer for gitlab-shell (port 22) so SSH git workflows work.
New config surface in conf/gke.cfg:
GITLAB_GLOBAL_STATIC_IP_NAME = git-knoe-dev (34.102.141.87)
SVC_KNOE_GLOBAL_STATIC_IP_NAME = svc-knoe (34.111.197.33)
SUPABASE_API_GLOBAL_STATIC_IP_NAME = supabase-api (34.120.221.5)
SUPABASE_STUDIO_GLOBAL_STATIC_IP_NAME = supabase-studio (136.110.189.6)
GITLAB_SSH_HOST = git-ssh.knoe.dev
GITLAB_SHELL_LOADBALANCER_IP = 34.106.243.154 (regional us-west3)
GITLAB_SHELL_EXTERNAL_TRAFFIC_POLICY = Local (default)
Why SSH lives on a separate hostname+IP: Google-managed certs require
a GCE global L7 IP; port 22 needs a regional Network LB. Those can not
share an IP on GCP, so git.knoe.dev stays on HTTPS and git-ssh.knoe.dev
takes SSH.
Wiring per surface:
- etc/init_gitlab.sh
* Added GITLAB_SSH_HOST (default git-ssh.<root> in k8s mode),
GITLAB_SHELL_LOADBALANCER_IP, GITLAB_SHELL_EXTERNAL_TRAFFIC_POLICY,
GITLAB_GLOBAL_STATIC_IP_NAME config keys.
* CR global.hosts.ssh now reads ${GITLAB_SSH_HOST}.
* gitlab-shell block conditionally renders service: {type:
LoadBalancer, loadBalancerIP, externalTrafficPolicy} when the LB IP
is set.
* GITLAB_GCE_TLS_ANNOTATIONS_YAML gains
kubernetes.io/ingress.global-static-ip-name: "<name>" when
GITLAB_GLOBAL_STATIC_IP_NAME is set.
- etc/init_kong.sh
* Added SVC_KNOE_GLOBAL_STATIC_IP_NAME var near other SERVICE_TLS_*
defaults.
* gce_tls_annotations heredoc gets the static-IP line appended on the
same condition.
- supabase/helm/render_supabase.py
* Two new _first(env, cfg, default) extractions for
SUPABASE_API_GLOBAL_STATIC_IP_NAME and
SUPABASE_STUDIO_GLOBAL_STATIC_IP_NAME near the existing managed-cert
/ frontend-config vars.
* Two new dict-spread blocks in the Kong + Studio ingress annotations
that emit the static-IP annotation only when mode=k8s and the value
is non-empty.
All three ingress surfaces already use kubectl apply (merge-friendly);
re-running init_gitlab.sh / init_kong.sh / deploy.sh (supabase step) is
enough to pick up the new annotation. The GKE LB controller will swap
each ingress's forwarding rule from the auto-generated k8s2-fr-*
reservation to the named reservation, then release the old ephemeral.
Ingress IPs change; DNS records need updating (TTL <=300s while
iterating). Google-managed cert for gitlab-managed-cert will briefly
Provision again during the swap; the other three were already
Provisioning.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Enables iterating on GitLab reconciles without a full deploy.sh run and
unblocks the end-to-end login flow:
- etc/init_gitlab.sh
* Pre-scan $@ for --config before sourcing prole_cfg.sh so
PROLE_DEPLOY_CFG is set for standalone invocations; previously
prole_cfg.sh auto-picked conf/k3d.cfg and leaked KUBECONTEXT=dev,
failing with `error: context "dev" does not exist`.
* _init_gitlab_resolve_secretref: bash mirror of
knoe/core/actions.py:_resolve_secretref_value so
secretref://google-oidc-client-* resolves from etc/secrets/ files
when run outside deploy.sh Python env-injection.
* Garage bucket array: drop -storage suffix from uploads/artifacts/
lfs/packages/dependency-proxy. Chart defaults for these have no
suffix, and the object_store block in the CR does not override
per-object bucket names; the prior mismatch caused first-login 500s
(NoSuchBucket on avatar PUT).
- deploy/gcp/gke/gitlab-google-oidc-secret.example.yaml
* Add discovery: true so omniauth-openid_connect fetches Google
.well-known/openid-configuration; fixes "Could not authenticate
from OpenIDConnect: No host info" on the callback.
- conf/gke.cfg
* GITLAB_WEBSERVICE_LIMITS_MEMORY 1800M -> 3Gi, REQUESTS_MEMORY
900M -> 2Gi, REQUESTS_CPU 200m -> 500m. Live pod was sitting at
1706Mi/1800M (99%) in OOMKilled loop.
- conf/port-mapping.cfg: add supabase + gitea forwards, fix postgres
namespace knoe-db -> knoe-db-0 for split-cluster CNPG layout.
Adds read-only diagnostics used to chase the above:
- etc/diag_gitlab_boot.sh, diag_gitlab_webservice_oom.sh, diag_gke_storage.sh
- etc/ensure_default_storage_class.sh + k8s/prole/storageclass-gcp-standard-hdd.yaml
(preflight + HDD-default SC manifest for SSD-quota-constrained GKE projects).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Summarize CLAUDE.md updates:
- Document both live GKE clusters as e2-standard-2 x3 in us-west3 and note stale app default in code.
- Call out DB cluster naming mismatch between conf (knoe-dev-cnpg-0) and code default (knoe-cnpg-0).
- Add SSD quota guidance: 300GB fully consumed by CNPG, so non-CNPG PVCs must use standard/pd-standard.
- Refresh reset script reference and quota section details to match current runtime state.
Co-authored-by: Junie <junie@jetbrains.com>
- Resolved context mismatches by updating DB references to APP where appropriate.
- Updated Supabase deployment to ensure proper handling of cross-cluster DB hosts.
- Simplified ingress logic by setting `split_frontdoor_to_db` to `false` for APP clusters lacking GCE Ingress support.
- Updated `init_kong.sh` to handle `include_gitea_host` flag for improved flexibility in Gitea ingress setup.
- Added comprehensive post-deploy endpoint summary in `deploy.sh` to display public services and external IPs.
- Enhanced kubectl command handling with role-specific context logic for better split-cluster support.
- Improved DB context logging across CNPG operations and milestones.
- Updated GKE configuration with new contexts and Grafana hostname.
- Added split-cluster detection with detailed app and DB context logging.
- Introduced robust DB connectivity checks for DNS resolution and TCP probes.
- Enabled automated cross-cluster DB ILB service creation and validation.
- Improved logging for DB migration failures with enhanced diagnostics and connectivity hints.
- remove hardcoded physical host assumptions in UI/init flows and rely on config-driven selectors\n- optimize GitLab reruns to skip unnecessary operator upgrades/long reconcile waits when unchanged\n- add and update regression/shell tests for fast-path and k8s context/ingress/garage behaviors
Co-authored-by: Junie <junie@jetbrains.com>
- Updated default paths, namespaces, and cluster names for k3d/dev configuration.
- Added `skip_names` set to exclude `prod.cfg` and `gcp.cfg` from override processing.
- Refined init parameters for Supabase, Kubernetes, and database deployments to align with dev-specific settings.
- Enhanced test coverage for excluded config files (`prod.cfg`, `gcp.cfg`) within `test_prole_conf.py`.
Rename env config files from conf/*/prole.cfg to conf/k3d.cfg, conf/k3s.cfg, and conf/gke.cfg. Update shell/Python loaders and etc/deploy scripts to resolve named configs cleanly while keeping legacy fallback behavior. Align k3s Ansible tasks, docs, and regression coverage with the new configuration layout.
Co-authored-by: Junie <junie@jetbrains.com>
Switch production config to k8s/GKE contexts and align service naming. Add immutable StatefulSet update fallback for Garage across k3d/k3s/k8s. Harden CNPG deploy and backup bootstrap paths, and update installer coverage for CNPG webhook and Garage common ops.
Co-authored-by: Junie <junie@jetbrains.com>
- Updated default paths, namespaces, and cluster names for k3d/dev configuration.
- Added `skip_names` set to exclude `prod.cfg` and `gcp.cfg` from override processing.
- Refined init parameters for Supabase, Kubernetes, and database deployments to align with dev-specific settings.
- Enhanced test coverage for excluded config files (`prod.cfg`, `gcp.cfg`) within `test_prole_conf.py`.
- _sanitize_sections_for_cfg(): wrap _expand_shell_path() with _cfgify_home_path() so
paths are stored as dollar-HOME/... instead of /home/user/... (Mac to myrddin breakage)
- _write_cfg(): pop KUBECONTEXT from globals_to_save so Mac k3d context never persists
- _write_cfg(): add CNPG Clusters to sections list so user-authored registry survives write-back
- milestone._get_script_env(): _GLOBAL_CFG_PROPAGATION_BLOCK excludes KUBECONTEXT, HOME,
USER from Global config propagation -- stops knoe.dev.prole.org injected into kubectl calls
- milestone._get_script_env(): env.pop KUBECONTEXT after k3s KUBECONFIG detection so
kubectl uses kubeconfig current-context (default) on k3s, not a stale Mac context name
- conf/service/prole.cfg: remove stale KUBECONTEXT = knoe.dev.prole.org from Global
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Introduced support for dynamic GitLab and Supabase domain and ingress configuration based on deployment mode (k8s vs local).
- Default GitLab domain and ingress class aligned with `prole.cfg` deployment settings.
- Expanded Supabase support:
- Added configurable API and Studio hostnames with default fallbacks.
- Introduced separate ingress class configuration for API and Studio.
- Standardized public URL normalization for both services.
- Updated tests to validate domain, ingress, and configuration behaviors across environments.
KNOE_IMAGE_REGISTRY auto-detection in cloudnative_pg._apply_manifest():
- Explicit config wins; falls back to gitlab-registry when GitLab ns exists;
final fallback to registry:2 in knoe-system. No longer raises when unset.
conf/service/prole.cfg: point KNOE_IMAGE_REGISTRY at gitlab-registry since
GitLab is already deployed on this cluster.
etc/init_registry.sh: add 'migrate' action that copies all images from
registry:2 to gitlab-registry via skopeo (prints commands if skopeo absent).
Run after GitLab is up, before decommissioning registry:2.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add [CNPG Clusters] section to prole.cfg as the authoritative source for
CNPG cluster definitions. The deploy pipeline reads this registry instead
of computing version-appended namespace names at runtime.
- conf/service/prole.cfg: add [CNPG Clusters] with knoe-db as identity
cluster; fix DATABASE_NAMESPACE=knoe-db (was knoe-db-18-140); add
KNOE_IMAGE_REGISTRY for the k3s in-cluster registry hostname
- deploy/opentofu/k3s/manifests/prole/knoe-db.yaml: replace hardcoded
registry.test-system image with templated KNOE_IMAGE_REGISTRY
- knoe/core/actions.py: add _cnpg_cluster_registry(), _cnpg_identity_cluster();
fix _initial_db_namespace() to use declared namespace without version suffix
- knoe/core/milestones.py: iterate over all declared clusters in Init and
Deploy milestones; per-cluster CNPG_IMAGE_NAME support
- knoe/core/ops/cloudnative_pg.py: resolve KNOE_IMAGE_REGISTRY and
CNPG_IMAGE_NAME overrides in _apply_manifest()
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- fix conf/prod/prole.cfg by replacing hardcoded /Users/chrisfu paths with /Users/chrisfu
- remove injected [update.sh] log lines and stray password artifacts so config is executable
- include latest pending updates across deployment config, UI/core flow, vault/network artifacts, and helper scripts
- separate default app and db cluster contexts (knoe-dev-0 and knoe-cnpg-0) across actions, ops flows, and installer config\n- update CNPG placement and GKE manifests to keep PostgreSQL on the db cluster while app services stay on the app cluster\n- add GCS-backed Barman ObjectStore manifest and cross-cluster Garage patch workflow\n- refresh UI cluster/database/service screens and monitoring wiring for dual-cluster operation\n- add reset/patch scripts, db context selection test coverage, and architecture/network documentation updates
Co-authored-by: Junie <junie@jetbrains.com>
- harden CNPG backup ObjectStore readiness fallback in statusless clusters by validating controller availability and expected spec
- improve backup/plugin reconciliation paths and related test coverage for barman-cloud flows
- include ongoing GKE/CNPG deployment and configuration updates from this checkpoint
Co-authored-by: Junie <junie@jetbrains.com>
UI screens
- database.py: fix mode detection to use env_key priority (prod→k8s, service→k3s) so stale DEPLOYMENT_MODE never overrides the user's chosen environment
- database.py: Registry status reads ARTIFACT_REGISTRY_AVAILABLE persisted by cluster screen; uses SERVICE_NAMESPACE for Artifact Registry repo name
- cluster.py: add Artifact Registry traffic light (amber→green/red) to prod section; _check_artifact_registry_async persists ARTIFACT_REGISTRY_AVAILABLE into Global cfg
- cluster.py: re-trigger Artifact Registry check after GKE cluster selection so the light re-evaluates once region is available from KUBECONTEXT
- cluster_nodes.py: fix TclError on Python 3.14 — pady=(2,0) tuple → pady=2 scalar
- __init__.py: seed knoe-system namespace when saved value is "default", not only when empty
- services.py: replace hardcoded "Prole DB" log string with dynamic cnpg_cluster name
Core ops
- cloudnative_pg.py: replace one-shot Barman plugin retry with 6-attempt loop; first cert-manager/x509 failure triggers rollout restart + 30 s CA propagation wait; subsequent failures back off up to 60 s per attempt
- cloudnative_pg.py: TLS CA CN now uses cluster_name instead of hardcoded "Prole CNPG CA"
- registry.py, garage_store.py: refactored into per-mode modules (k3d/k3s/k8s registry and garage store, shared _garage_common)
Deploy / config
- deploy/gcp/gke/knoe-db.yaml: GKE-specific CNPG cluster manifest (rw/ro/r on separate nodes with premium-rwo storage)
- etc/init_common_services.sh, modes/k8s/knoe-db/.version: updated for current deploy
- kong-deployment.yaml: updated manifest
Tests
- test_cluster_nodes_render_smoke.py: add pack/grid, winfo_children, winfo_reqheight, update_idletasks, grid_slaves to dummy widgets; monkeypatch tk.Label so CNPG placement render completes without a real Tkinter root
Co-authored-by: Junie <junie@jetbrains.com>
## GCP / Cluster Environment Screen
- Auto-populate Cloud tab from conf/prod/gcp.cfg on screen open (org_id,
billing_account, billing_project, project_id)
- gcloud auth validity checked on screen startup; friendly modal dialog
streams gcloud auth login output live so user never leaves the app
- Live GKE cluster browser: fetches clusters via gcloud container clusters
list, displays with checkmark selector, auto-selects saved cluster
- Selecting a cluster runs get-credentials, sets KUBECONFIG/KUBECONTEXT,
and syncs the region dropdown to the selected cluster's location
- Region dropdown populated live from gcloud compute regions list with
checkmark on currently selected region; graceful fallback when offline
- New 'GCP Storage' tab with workload->StorageClass mapping (CNPG->premium-rwo,
Redis/Monitoring->standard-rwo, Garage->garage-hdd) and Fetch from Cluster
- Provider readonly field styled correctly (no solid-black on macOS)
- Stale prole.cfg/conf/prole.cfg symlinks removed; all config I/O now
resolves env-specific paths via prole_conf.entrypoint_path()
## GKE Autopilot Compatibility (Common Services)
- Synology iSCSI StorageClass and static PVs guarded behind PROLE_MODE!=k8s
in init_openbao.sh (GKE Autopilot forbids hostPath/iSCSI volumes)
- In-cluster Docker registry (hostPath) skipped in k8s mode; GCP Artifact
Registry used instead
- Kong renamed knoe-svc-kong in k8s mode; all health-check kubectl calls in
init_common_services.sh and status_common_services.sh updated accordingly
- DNS endpoints switched from *.prole.org to *.knoe.dev in k8s mode
(api.knoe.dev, git.knoe.dev, svc.knoe.dev); ingress uses gce class
- New GKE-clean Kong manifests under deploy/opentofu/k8s/manifests/prole/:
no k3s node affinity, explicit Autopilot resource requests/limits
## Garage S3 Store (GKE)
- New garage-statefulset-gcp.yaml targeting garage-hdd StorageClass
(pd-standard, avoids SSD_TOTAL_GB quota exhaustion in us-west3)
- New storageclass-gcp-hdd.yaml (pd-standard, Retain, WaitForFirstConsumer)
- GCP StorageClass manifests skipped on re-runs (Autopilot built-ins are
immutable; skip-if-exists guard added)
- PVC deletion guard extended to cover any storageClass (not just synology)
so stale claims are cleaned before StatefulSet recreation
## Topology (GKE Autopilot)
- DaemonSet collector skipped in prod mode (forbidden in kube-system by
GKE Warden); Kubernetes-only node facts path used instead
- All ready GKE nodes assumed cnpg-eligible and monitoring-eligible without
taint/synology-mount checks (skip_collector + assume_nodes_eligible flags)
## KUBECONFIG / kubectl (k8s mode)
- actions.py: new elif mode==k8s branch sets KUBECONFIG=~/.kube/config
and injects KUBECONTEXT from prole_cfg_data into script env
- _build_kubectl_cmd falls back to Global.KUBECONTEXT when
init_cluster.selected_kubectx is empty
- _activate_selected_gke_cluster persists KUBECONFIG/KUBECONTEXT to
prole_cfg_data and saves prole.cfg immediately after get-credentials
## Database Build Screen (GKE)
- Registry display shows correct Artifact Registry URL
(<region>-docker.pkg.dev/<project>/<namespace>/knoe-db) in green
- Build+push: gcloud auth configure-docker, auto-creates AR repository
named after SERVICE_NAMESPACE (e.g. knoe-system) if missing, then
docker tag + push; falls back to gcr.io if region unavailable
- GCP config loaded from conf/prod/gcp.cfg on every screen entry;
keys normalised to lowercase so project_id lookup is always consistent
## Config / Namespace persistence
- prole_conf.py activate_environment: symlink creation removed; sets
CLUSTER_ENV env-var so all subsequent calls resolve correct env directory
- knoe/ui/screens/__init__.py: startup config load uses entrypoint_path()
instead of hardcoded conf/prole.cfg; seeds SERVICE_NAMESPACE=knoe-system
for managed envs so Common Services never defaults to 'default'
- cfg.py _save_prole_cfg: saves to env-specific path via entrypoint_path()
- etc/prole_cfg.sh: removed all ln -snf symlink creation
Co-authored-by: Junie <junie@jetbrains.com>
