chrisfu/prole
1
0
Fork 0
mirror of https://github.com/dredx/prole.git synced 2026-09-23 10:13:58 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity
dba8a2d1dc
prole/knoe/ui/screens/cluster.py
chrisfu dba8a2d1dc feat(installer): TDD stabilization for k3d install path; dual-cluster GKE TUI 
Junie's session targeted the prompt "stabilize ./install.py -c conf/k3d.cfg
using strict TDD" — broad installer-side work, not the k3d-mirror Phase 3
brief I had filed (which she didn't pick up; phase-3 brief stays open). All
750 installer tests pass post-change.

What Junie produced:

  install.py                          (NEW) Top-level CLI entry point. Was
                                            imagined by the prompt but didn't
                                            exist; this commit makes it real.
  knoe/deployment.py                  (NEW) `KnoeDeployment` orchestrator for
                                            the k3s service-mode deploy pipeline.
                                            Wraps Ansible kubeconfig fetch,
                                            opentofu apply, init_*.sh post-apply
                                            scripts, and (optionally) supabase/
                                            deploy.sh.
  knoe/ui/screens/cluster.py          Dual-cluster GKE kubecontext UI: prod env
  knoe/ui/screens/cfg.py              now shows separate "App Cluster:" and
                                       "DB Cluster:" dropdowns instead of a
                                       single "Kubernetes Context:" combo.
                                       New _app_kubectx_combo + _db_kubectx_combo
                                       widgets; new app/db_cluster_kubecontext
                                       tk.StringVars.
  knoe/core/{actions,env,milestones}.py
  knoe/core/ops/storage.py
  knoe/config.py, knoe/knoe_conf.py   Plumbing changes for the dual-cluster
                                       kubecontext flow + storage-class topology
                                       detection cleanup.
  knoe/tools/cleanup_cnpg_storage.py  (NEW) Stand-alone cleanup utility.
  tools/dashboard.sh                  (NEW) Dashboard helper.
  conf/knoe.cfg                       (NEW) Master cfg generated by knoe_conf.
  conf/dev/                           (NEW) Dev-mode cfg directory.
  conf/port-mapping.cfg               Port mapping tweaks for k3d.
  tests/installer/* (8 files)         New + extended tests for the dual-cluster
  tests/test_database_options.py      TUI, kubecontext save flow, storage ops,
                                       topology detection, deploy helpers,
                                       database-options screen.

Issues found in Junie's working state and fixed here:

  1. install.py was a 11-line import shim with no shebang, no `chmod +x`,
     no `if __name__ == '__main__'` block. `./install.py -c conf/k3d.cfg`
     returned `Permission denied` and `python install.py` did nothing.
     Added `#!/usr/bin/env python3`, `chmod +x`, and a __main__ block
     that delegates to `knoe.ui.screens.main()`. `./install.py --help`
     now prints the canonical argparse help.

  2. knoe/deployment.py had FIVE `subprocess.run()` call sites with no
     `timeout=` argument (`_run_script`, `_run_cmd`, the Ansible playbook
     fetch, `tofu init`, `tofu apply`). A hung child process — typical
     failure mode is a script waiting on stdin or a stalled network
     call — would lock up the installer indefinitely. Added timeouts:
       - Ansible kubeconfig fetch: 120s
       - tofu init: 300s
       - tofu apply, _run_script, _run_cmd: bounded by new module
         constant `_MILESTONE_TIMEOUT` (default 1800s = 30 min, override
         via `KNOE_MILESTONE_TIMEOUT_SECONDS` env var).
     `subprocess.TimeoutExpired` is caught explicitly; on timeout the
     run helpers return exit code 124 (conventional timeout code).

  3. `conf/k3d.cfg` was corrupted with MagicMock string-reprs on disk:
        KNOE_CONF = <MagicMock name='Canvas().tk.call().strip()' id='4743999712'>
        argocd.node_selector = <MagicMock name='mock.StringVar().get().strip()' id='...'>
     Likely path: Junie ran `./install.py -c conf/k3d.cfg` interactively
     in a non-Tk environment (or with a partially-mocked widget set) and
     the installer's "save current state" path wrote the mock-objects'
     `__repr__` strings into the cfg file. This commit reverts the cfg
     to its pre-Junie state. **Followup: harden the cfg save path
     against non-string widget values** — track separately.

  4. The corrupted cfg caused the installer to call `os.makedirs()` on
     the mock-string values, producing 10 directories on disk literally
     named `<MagicMock name='Canvas().tk.call().strip()' id='4733210304'>/`
     etc., with 5–86 files of install artifacts inside each. Removed.

The "final step is timing out" the user reported was almost certainly
issue #2 above: install.py walked the milestone pipeline, hit one of
the unbounded subprocess.run calls, and the wrapped command (probably
supabase/deploy.sh, which Junie was reading for context when her
session timed out) hung. With the timeouts in place that path now
exits cleanly with rc=124 instead of locking up.

Verification:
  - pytest tests/installer/ -q                                   750 passed in ~25s
  - python3 -c "import knoe.deployment"                          imports clean
  - ./install.py --help                                          prints argparse help
  - find . -maxdepth 1 -type d -name '<MagicMock*' | wc -l       0
  - head -7 conf/k3d.cfg                                          clean (no MagicMock)

Out of scope for this commit (followups):
  - The cfg save-path that wrote mock-objects-as-strings (issue #3 root cause).
    Reproducer: launch the installer in an env where Tk widget vars are
    `unittest.mock.MagicMock` instances. The cfg save code should refuse to
    serialize non-str values rather than calling `str()` on a MagicMock.
  - The k3d-mirror Phase 3 brief (`docs/plans/junie/k3d-knoe-auth-pod-deploy.md`)
    is still open — Junie picked a different prompt this round.

Co-authored-by: Junie <junie@jetbrains.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-02 23:51:14 -07:00

4034 lines
162 KiB
Python
Raw Blame History

"""Cluster lifecycle management (k3d / k3s / k8s)."""
import os
import re
import subprocess
import sys
import threading
import time
import tkinter as tk
from pathlib import Path
from tkinter import ttk, messagebox, filedialog
import json
from knoe import knoe_conf
from knoe import screen as ui
from knoe.config import (
_collect_cfg_vars_from_data,
_encrypt_cfg_secret,
_expand_path_expr,
_merge_kubeconfig,
_parse_gcp_cfg,
_write_k3s_kubeconfig,
)
from knoe.core.actions import _reset_k3s_namespace
from knoe.core.env import (
PROJECT_ROOT,
_default_opentofu_pipeline_url,
_deployment_mode_from_env,
_deployment_target_label,
_find_kubeconfig_file,
_k3d_knoe_data_volume_args,
_normalize_cluster_env,
_normalize_k3s_token,
_resolve_k3s_connection as _resolve_k3s_connection_fn,
_safe_str,
)
from knoe.core.prod_config import ProdConfigApi, build_config, to_api_payload
class ClusterScreenMixin:
"""Cluster lifecycle management (k3d / k3s / k8s)."""
def _render_init_cluster_page(self):
# Letterhead at top right
content_width = self.bg_canvas.winfo_width() or 975
right_margin = content_width - 48
ui.canvas_text(
self,
right_margin,
40,
"knoe.dev",
fill="#6e6e73",
font=("SF Pro Text", 32, "bold"),
anchor="ne",
)
ui.canvas_text(
self,
right_margin,
85,
"infrastructure.auto()",
fill="#6e6e73",
font=("SF Pro Text", 18),
anchor="ne",
)
# Tighten vertical spacing on this screen so the main status area (traffic lights +
# console output) has enough room in the fixed-size window.
self._render_title("Cluster Environment", y=110)
self._render_paragraph(
"Select the cluster environment for this session."
" Dev runs a local k3d cluster on this machine."
" Service and Prod connect to remote clusters using the server URL and token below.",
y=155,
)
# Cluster Selection (Radio Buttons)
x_label = 48
y = 210
self._canvas_items.append(
ui.canvas_text(
self,
x_label,
y,
"Select Environment:",
fill="black",
font=("SF Pro Text", 14, "bold"),
)
)
y += 36
cluster_options = [("dev", "Dev"), ("service", "Service"), ("prod", "Prod")]
# We need to trace cluster_env if not already traced
if not hasattr(self, "_cluster_env_trace"):
self._cluster_env_trace = self.cluster_env.trace_add(
"write", self._on_cluster_env_change
)
rb_x = x_label + 20
rb_gap = 150
for idx, (val, name) in enumerate(cluster_options):
rb = tk.Radiobutton(
self.bg_canvas,
text=name,
variable=self.cluster_env,
value=val,
bg="white",
fg="black",
activebackground="white",
selectcolor="white",
font=("SF Pro Text", 11),
)
rb_window = self.bg_canvas.create_window(
rb_x + idx * rb_gap, y, window=rb, anchor="nw"
)
self._canvas_items.append(rb_window)
self._overlay_widgets.append(rb)
selected_env_key = self._cluster_env_key()
y += 36
if selected_env_key == "dev":
# Combined Select / Add Cluster widget
self._canvas_items.append(
ui.canvas_text(
self,
x_label,
y,
"Cluster:",
fill="black",
font=("SF Pro Text", 12, "bold"),
)
)
y += 28
clusters = self._get_k3d_cluster_list()
combo = ttk.Combobox(
self.bg_canvas,
textvariable=self.selected_k3d_cluster,
values=clusters,
width=30,
)
if not self.selected_k3d_cluster.get() and clusters:
self.selected_k3d_cluster.set(clusters[0])
combo.bind(
"<<ComboboxSelected>>", lambda _evt: self._on_k3d_cluster_select()
)
combo_win = self.bg_canvas.create_window(
x_label + 20, y, window=combo, anchor="nw"
)
self._canvas_items.append(combo_win)
self._overlay_widgets.append(combo)
add_btn = tk.Button(
self.bg_canvas,
text="Add",
command=self._on_create_k3d_cluster,
bg="#F5F5DC",
fg="black",
activebackground="#E5E5D5",
activeforeground="black",
highlightbackground="#F5F5DC",
highlightthickness=0,
relief="flat",
font=("SF Pro Text", 10),
padx=10,
)
add_win = self.bg_canvas.create_window(
x_label + 290, y - 4, window=add_btn, anchor="nw"
)
self._canvas_items.append(add_win)
self._overlay_widgets.append(add_btn)
delete_btn = tk.Button(
self.bg_canvas,
text="Delete",
command=self._on_delete_k3d_cluster,
bg="#F5F5DC",
fg="#cc0000",
activebackground="#E5E5D5",
activeforeground="#cc0000",
highlightbackground="#F5F5DC",
highlightthickness=0,
relief="flat",
font=("SF Pro Text", 10),
padx=10,
)
delete_win = self.bg_canvas.create_window(
x_label + 350, y - 4, window=delete_btn, anchor="nw"
)
self._canvas_items.append(delete_win)
self._overlay_widgets.append(delete_btn)
y += 40
else:
# Service / Prod Section
values = self._get_kubectx_list()
if selected_env_key == "prod":
# ---- k8s / prod mode: explicit App + DB cluster dropdowns ----
self._canvas_items.append(
ui.canvas_text(
self,
x_label,
y,
"App Cluster:",
fill="black",
font=("SF Pro Text", 12, "bold"),
)
)
app_combo = ttk.Combobox(
self.bg_canvas,
textvariable=self.app_cluster_kubecontext,
values=values,
state="readonly",
width=40,
)
app_combo.bind("<<ComboboxSelected>>", self._on_dual_kubectx_select)
app_combo_win = self.bg_canvas.create_window(
x_label + 140, y - 6, window=app_combo, anchor="nw"
)
self._canvas_items.append(app_combo_win)
self._overlay_widgets.append(app_combo)
self._app_kubectx_combo = app_combo
y += 34
self._canvas_items.append(
ui.canvas_text(
self,
x_label,
y,
"DB Cluster:",
fill="black",
font=("SF Pro Text", 12, "bold"),
)
)
db_combo = ttk.Combobox(
self.bg_canvas,
textvariable=self.db_cluster_kubecontext,
values=values,
state="readonly",
width=40,
)
db_combo.bind("<<ComboboxSelected>>", self._on_dual_kubectx_select)
db_combo_win = self.bg_canvas.create_window(
x_label + 140, y - 6, window=db_combo, anchor="nw"
)
self._canvas_items.append(db_combo_win)
self._overlay_widgets.append(db_combo)
self._db_kubectx_combo = db_combo
dual_apply_btn = tk.Button(
self.bg_canvas,
text="Apply",
command=self._on_kubectx_apply_dual,
bg="#F5F5DC",
fg="black",
activebackground="#E5E5D5",
highlightbackground="#F5F5DC",
highlightthickness=0,
relief="flat",
font=("SF Pro Text", 10),
padx=10,
state="disabled",
)
dual_apply_win = self.bg_canvas.create_window(
x_label + 450, y - 10, window=dual_apply_btn, anchor="nw"
)
self._canvas_items.append(dual_apply_win)
self._overlay_widgets.append(dual_apply_btn)
self._kubectx_apply_btn = dual_apply_btn
self._kubectx_apply_btn_canvas_window = dual_apply_win
self._update_kubectx_apply_button()
y += 34
else:
# ---- Service / k3s mode: single kubectx combobox ----
self._canvas_items.append(
ui.canvas_text(
self,
x_label,
y,
"Kubernetes Context:",
fill="black",
font=("SF Pro Text", 12, "bold"),
)
)
combo = ttk.Combobox(
self.bg_canvas,
textvariable=self.selected_kubectx,
values=values,
state="readonly",
width=40,
)
combo.bind("<<ComboboxSelected>>", self._on_kubectx_select)
if not self.selected_kubectx.get() and values:
# Default to the current context if it exists in the list; otherwise pick
# a sensible default without switching immediately.
current_ctx = (getattr(self, "_kubectx_applied", "") or "").strip()
if not current_ctx:
current_ctx = (self._current_kubectl_context() or "").strip()
if current_ctx:
self._kubectx_applied = current_ctx
if current_ctx and current_ctx in values:
self.selected_kubectx.set(current_ctx)
elif "knoe-k3s" in values:
self.selected_kubectx.set("knoe-k3s")
else:
self.selected_kubectx.set(values[0])
combo_win = self.bg_canvas.create_window(
x_label + 270, y - 6, window=combo, anchor="nw"
)
self._canvas_items.append(combo_win)
self._overlay_widgets.append(combo)
# Keep references for UI tests / layout verification.
self._kubectx_combo = combo
self._kubectx_combo_canvas_window = combo_win
ui.canvas_text(
self,
x_label + 580,
y + 2,
"via kubectx",
fill="#6e6e73",
font=("SF Pro Text", 10),
)
apply_btn = tk.Button(
self.bg_canvas,
text="Apply",
command=self._on_kubectx_apply,
bg="#F5F5DC",
fg="black",
activebackground="#E5E5D5",
highlightbackground="#F5F5DC",
highlightthickness=0,
relief="flat",
font=("SF Pro Text", 10),
padx=10,
state="disabled",
)
apply_win = self.bg_canvas.create_window(
x_label + 670, y - 10, window=apply_btn, anchor="nw"
)
self._canvas_items.append(apply_win)
self._overlay_widgets.append(apply_btn)
# Keep references for UI tests / layout verification.
self._kubectx_apply_btn = apply_btn
self._kubectx_apply_btn_canvas_window = apply_win
self._update_kubectx_apply_button()
y += 34
if selected_env_key == "prod":
# Staging directory input
self._canvas_items.append(
ui.canvas_text(
self,
x_label,
y,
"Local Artifact Staging Directory:",
fill="black",
font=("SF Pro Text", 12, "bold"),
)
)
entry = tk.Entry(
self.bg_canvas,
textvariable=self.prod_artifacts_path,
width=50,
bg="white",
fg="black",
insertbackground="black",
highlightbackground="#CCCCCC",
highlightthickness=1,
relief="flat",
font=("SF Pro Text", 11),
)
entry_win = self.bg_canvas.create_window(
x_label + 260, y - 6, window=entry, anchor="nw"
)
self._canvas_items.append(entry_win)
self._overlay_widgets.append(entry)
browse_btn = tk.Button(
self.bg_canvas,
text="Browse...",
command=lambda: self.prod_artifacts_path.set(
filedialog.askdirectory() or self.prod_artifacts_path.get()
),
)
browse_win = self.bg_canvas.create_window(
x_label + 770, y - 8, window=browse_btn, anchor="nw"
)
self._canvas_items.append(browse_win)
self._overlay_widgets.append(browse_btn)
y += 34
ui.canvas_text(
self,
x_label + 20,
y,
"(Used by etc/deploy_pipeline.sh --mode gcp)",
fill="#6e6e73",
font=("SF Pro Text", 10),
)
y += 40
self._ensure_prod_config_state()
self._canvas_items.append(
ui.canvas_text(
self,
x_label,
y,
"Production Configuration",
fill="black",
font=("SF Pro Text", 12, "bold"),
)
)
y += 24
notebook = ttk.Notebook(self.bg_canvas)
cloud_tab = tk.Frame(notebook, bg="white")
db_tab = tk.Frame(notebook, bg="white")
backup_tab = tk.Frame(notebook, bg="white")
auth_tab = tk.Frame(notebook, bg="white")
migration_tab = tk.Frame(notebook, bg="white")
plan_tab = tk.Frame(notebook, bg="white")
storage_tab = tk.Frame(notebook, bg="white")
notebook.add(cloud_tab, text="Cloud")
notebook.add(db_tab, text="Database")
notebook.add(backup_tab, text="Backup + Storage")
notebook.add(storage_tab, text="GCP Storage")
notebook.add(auth_tab, text="Auth + Routing")
notebook.add(migration_tab, text="Migration")
notebook.add(plan_tab, text="Plan / Apply")
for frame in [cloud_tab, db_tab, backup_tab, storage_tab, auth_tab, migration_tab, plan_tab]:
frame.columnconfigure(1, weight=1)
def _add_row(
frame: tk.Frame,
row: int,
label: str,
key: str,
width: int = 40,
readonly: bool = False,
) -> int:
tk.Label(
frame,
text=label,
bg="white",
fg="black",
font=("SF Pro Text", 10),
anchor="w",
).grid(row=row, column=0, sticky="w", padx=8, pady=3)
entry = tk.Entry(
frame,
textvariable=self.prod_form_vars[key],
width=width,
bg="white",
fg="black",
insertbackground="black",
highlightbackground="#CCCCCC",
highlightthickness=1,
relief="flat",
font=("SF Pro Text", 10),
)
if readonly:
entry.configure(state="readonly", readonlybackground="#f0f0f0", fg="#555555")
entry.grid(row=row, column=1, sticky="ew", padx=8, pady=3)
return row + 1
cloud_row = 0
cloud_row = _add_row(cloud_tab, cloud_row, "Provider", "cloud.provider", readonly=True)
cloud_row = _add_row(cloud_tab, cloud_row, "GCP Project ID", "cloud.projectId")
# --- Region: live-populated Combobox ---
tk.Label(
cloud_tab, text="Region", bg="white", fg="black",
font=("SF Pro Text", 10), anchor="w",
).grid(row=cloud_row, column=0, sticky="w", padx=8, pady=3)
_region_frame = tk.Frame(cloud_tab, bg="white")
_region_frame.grid(row=cloud_row, column=1, sticky="ew", padx=8, pady=3)
_region_frame.columnconfigure(0, weight=1)
import tkinter.ttk as _ttk
self._gcp_region_combo = _ttk.Combobox(
_region_frame,
textvariable=self.prod_form_vars["cloud.region"],
width=38,
font=("SF Pro Text", 10),
state="normal",
)
self._gcp_region_combo.grid(row=0, column=0, sticky="ew")
self._gcp_region_status_var = tk.StringVar(value="")
tk.Label(
_region_frame,
textvariable=self._gcp_region_status_var,
bg="white", fg="#888888",
font=("SF Pro Text", 8),
).grid(row=1, column=0, sticky="w")
cloud_row += 1
# --- GKE Cluster Browser ---
tk.Label(
cloud_tab, text="GKE Clusters", bg="white", fg="black",
font=("SF Pro Text", 10), anchor="w",
).grid(row=cloud_row, column=0, sticky="nw", padx=8, pady=3)
_cluster_frame = tk.Frame(cloud_tab, bg="white")
_cluster_frame.grid(row=cloud_row, column=1, sticky="ew", padx=8, pady=3)
_cluster_frame.columnconfigure(0, weight=1)
self._gke_cluster_listbox = tk.Listbox(
_cluster_frame,
height=4,
font=("SF Pro Mono", 10),
bg="#f5f5f0",
fg="#1a1a1a",
selectbackground="#1558b0",
selectforeground="white",
relief="solid",
highlightthickness=1,
highlightbackground="#CCCCCC",
activestyle="none",
exportselection=False,
)
self._gke_cluster_listbox.grid(row=0, column=0, sticky="ew")
_gke_scrollbar = tk.Scrollbar(
_cluster_frame, orient="vertical",
command=self._gke_cluster_listbox.yview,
)
_gke_scrollbar.grid(row=0, column=1, sticky="ns")
self._gke_cluster_listbox.configure(yscrollcommand=_gke_scrollbar.set)
self._gke_cluster_status_var = tk.StringVar(value="")
tk.Label(
_cluster_frame,
textvariable=self._gke_cluster_status_var,
bg="white", fg="#888888",
font=("SF Pro Text", 8),
).grid(row=1, column=0, columnspan=2, sticky="w")
self._gke_kubeconfig_status_var = tk.StringVar(value="")
tk.Label(
_cluster_frame,
textvariable=self._gke_kubeconfig_status_var,
bg="white", fg="#1558b0",
font=("SF Pro Text", 8),
wraplength=360, justify="left",
).grid(row=2, column=0, columnspan=2, sticky="w")
cloud_row += 1
def _on_gke_cluster_select(event=None):
self._activate_selected_gke_cluster()
self._gke_cluster_listbox.bind("<<ListboxSelect>>", _on_gke_cluster_select)
cloud_row = _add_row(cloud_tab, cloud_row, "VPC Mode", "cloud.vpcMode")
cloud_row = _add_row(cloud_tab, cloud_row, "VPC Name", "cloud.vpcName")
cloud_row = _add_row(cloud_tab, cloud_row, "Subnet Name", "cloud.subnetName")
cloud_row = _add_row(
cloud_tab, cloud_row, "Artifact Registry", "cloud.artifactRegistry"
)
cloud_row = _add_row(cloud_tab, cloud_row, "DNS Zone", "cloud.dnsZone")
# --- Read-only GCP metadata from gcp.cfg ---
self._gcp_org_id_var = tk.StringVar(value="")
self._gcp_billing_acct_var = tk.StringVar(value="")
self._gcp_billing_proj_var = tk.StringVar(value="")
self._gcloud_auth_status_var = tk.StringVar(value="⏳ checking…")
def _add_info_row(row: int, label: str, var: tk.StringVar) -> int:
tk.Label(
cloud_tab, text=label, bg="white", fg="#555555",
font=("SF Pro Text", 9), anchor="w",
).grid(row=row, column=0, sticky="w", padx=8, pady=2)
tk.Entry(
cloud_tab, textvariable=var, state="readonly", width=40,
bg="#f5f5f5", fg="#333333", relief="flat",
highlightbackground="#DDDDDD", highlightthickness=1,
font=("SF Pro Text", 9),
).grid(row=row, column=1, sticky="ew", padx=8, pady=2)
return row + 1
cloud_row = _add_info_row(cloud_row, "Org ID", self._gcp_org_id_var)
cloud_row = _add_info_row(cloud_row, "Billing Account", self._gcp_billing_acct_var)
cloud_row = _add_info_row(cloud_row, "Billing Project", self._gcp_billing_proj_var)
# gcloud auth status
tk.Label(
cloud_tab, text="gcloud auth", bg="white", fg="#555555",
font=("SF Pro Text", 9), anchor="w",
).grid(row=cloud_row, column=0, sticky="w", padx=8, pady=2)
tk.Label(
cloud_tab, textvariable=self._gcloud_auth_status_var,
bg="white", fg="#1a73e8", font=("SF Pro Text", 9), anchor="w",
).grid(row=cloud_row, column=1, sticky="w", padx=8, pady=2)
cloud_row += 1
# Populate info vars from values already loaded in _ensure_prod_config_state
_gcp = getattr(self, "_gcp_cfg_values", {}) or {}
self._gcp_org_id_var.set(_gcp.get("ORG_ID", ""))
self._gcp_billing_acct_var.set(_gcp.get("BILLING_ACCOUNT", ""))
self._gcp_billing_proj_var.set(_gcp.get("BILLING_PROJECT", ""))
def _on_load_gcp_cfg():
import glob as _glob
conf_dir = self._resolve_knoe_conf_dir()
found = sorted(_glob.glob(str(conf_dir / "**" / "gcp.cfg"), recursive=True))
prod_path = conf_dir / "prod" / "gcp.cfg"
chosen = prod_path if prod_path.exists() else (Path(found[0]) if found else None)
if not chosen:
import tkinter.messagebox as _mb
_mb.showwarning(
"GCP Setup",
"No gcp.cfg found under conf/\n\n"
"Run: python3 etc/config.py --mode k8s --provider gcp",
)
return
loaded = _parse_gcp_cfg(chosen)
if not loaded:
import tkinter.messagebox as _mb
_mb.showwarning("GCP Setup", f"No values found in:\n{chosen}")
return
upper = {k.upper(): v for k, v in loaded.items()}
self.knoe_cfg_data.setdefault("GCP", {}).update(upper)
self._gcp_cfg_values = dict(self.knoe_cfg_data["GCP"])
if upper.get("PROJECT_ID"):
self.prod_form_vars["cloud.projectId"].set(upper["PROJECT_ID"])
if upper.get("REGION"):
self.prod_form_vars["cloud.region"].set(upper["REGION"])
self._gcp_org_id_var.set(upper.get("ORG_ID", ""))
self._gcp_billing_acct_var.set(upper.get("BILLING_ACCOUNT", ""))
self._gcp_billing_proj_var.set(upper.get("BILLING_PROJECT", ""))
def _on_refresh_auth():
self._gcloud_auth_status_var.set("⏳ refreshing…")
self._refresh_gcloud_auth_status()
def _on_refresh_clusters():
self._gke_cluster_status_var.set("⏳ loading clusters…")
self._fetch_gke_clusters()
btn_frame = tk.Frame(cloud_tab, bg="white")
btn_frame.grid(row=cloud_row, column=0, columnspan=2, sticky="w", padx=8, pady=(6, 2))
tk.Button(
btn_frame, text="Reload gcp.cfg", command=_on_load_gcp_cfg,
font=("SF Pro Text", 10),
).pack(side="left", padx=(0, 6))
tk.Button(
btn_frame, text="Refresh Auth", command=_on_refresh_auth,
font=("SF Pro Text", 10),
).pack(side="left", padx=(0, 6))
tk.Button(
btn_frame, text="Refresh Clusters", command=_on_refresh_clusters,
font=("SF Pro Text", 10),
).pack(side="left")
# Kick off gcloud auth check, region fetch, and cluster list on screen open
self.safe_after(self._refresh_gcloud_auth_status, delay=150)
self.safe_after(self._fetch_gcp_regions, delay=300)
self.safe_after(self._fetch_gke_clusters, delay=600)
db_row = 0
db_row = _add_row(
db_tab, db_row, "Namespace", "kubernetes.namespace", readonly=True
)
db_row = _add_row(db_tab, db_row, "CNPG Cluster", "database.clusterName")
db_row = _add_row(db_tab, db_row, "Postgres Version", "database.postgresVersion")
db_row = _add_row(db_tab, db_row, "Instances", "database.instances")
db_row = _add_row(db_tab, db_row, "Node Boot Disk Type", "init_cluster.db_boot_disk_type")
db_row = _add_row(db_tab, db_row, "Node Boot Disk Size (GB)", "init_cluster.db_boot_disk_size_gb")
db_row = _add_row(db_tab, db_row, "Storage Class", "database.storageClass")
db_row = _add_row(db_tab, db_row, "Storage Size (Gi)", "database.storageSizeGi")
db_row = _add_row(db_tab, db_row, "App Database", "database.appDatabase")
db_row = _add_row(db_tab, db_row, "Meta Database", "database.metaDatabase")
db_row = _add_row(db_tab, db_row, "App User", "database.appUser")
_add_row(db_tab, db_row, "Admin User", "database.adminUser")
backup_row = 0
backup_row = _add_row(
backup_tab, backup_row, "Backup Bucket", "backups.backupBucket"
)
backup_row = _add_row(backup_tab, backup_row, "WAL Bucket", "backups.walBucket")
_add_row(backup_tab, backup_row, "Retention Days", "backups.retentionDays")
auth_row = 0
auth_row = _add_row(auth_tab, auth_row, "Auth Provider", "auth.provider", readonly=True)
auth_row = _add_row(auth_tab, auth_row, "Issuer", "auth.issuer")
auth_row = _add_row(auth_tab, auth_row, "OIDC Client ID Ref", "auth.clientId")
auth_row = _add_row(auth_tab, auth_row, "OIDC Client Secret Ref", "auth.clientSecret")
auth_row = _add_row(
auth_tab,
auth_row,
"Bootstrap Admin Email",
"auth.bootstrapAdminEmail",
)
auth_row = _add_row(auth_tab, auth_row, "Frontdoor Host", "routing.frontdoorHost")
auth_row = _add_row(auth_tab, auth_row, "Platform Domain", "routing.platformDomain")
_add_row(auth_tab, auth_row, "TLS Mode", "routing.tlsMode")
migration_row = 0
migration_row = _add_row(
migration_tab,
migration_row,
"Source Environment",
"migration.sourceEnvironment",
)
migration_row = _add_row(migration_tab, migration_row, "Mode", "migration.mode")
migration_row = _add_row(
migration_tab, migration_row, "Source Host", "migration.sourceHost"
)
migration_row = _add_row(
migration_tab, migration_row, "Source Port", "migration.sourcePort"
)
migration_row = _add_row(
migration_tab,
migration_row,
"Source Database",
"migration.sourceDatabase",
)
migration_row = _add_row(
migration_tab, migration_row, "Source User", "migration.sourceUser"
)
migration_row = _add_row(
migration_tab,
migration_row,
"Source Password Ref",
"migration.sourcePasswordRef",
)
tk.Checkbutton(
migration_tab,
text="Continuous until cutover",
variable=self.prod_bool_vars["migration.continuousUntilCutover"],
bg="white",
fg="black",
activebackground="white",
selectcolor="white",
font=("SF Pro Text", 10),
).grid(
row=migration_row,
column=0,
columnspan=2,
sticky="w",
padx=8,
pady=6,
)
# ── GCP Storage mapping tab ─────────────────────────────────────
_sc_defaults = {
"storage.cnpg": "premium-rwo",
"storage.monitoring": "standard-rwo",
"storage.garage": "standard-rwo",
}
for _k, _v in _sc_defaults.items():
if _k not in self.prod_form_vars:
self.prod_form_vars[_k] = tk.StringVar(value=_v)
tk.Label(
storage_tab,
text="Map each workload category to a GCP StorageClass."
" Click 'Fetch from Cluster' to load available classes.",
bg="white", fg="#444", anchor="w",
font=("SF Pro Text", 9),
wraplength=820,
).grid(row=0, column=0, columnspan=3, sticky="w", padx=8, pady=(6, 2))
_sc_hdr_font = ("SF Pro Text", 9, "bold")
for _col, _hdr in enumerate(["Workload", "StorageClass", "Notes"]):
tk.Label(
storage_tab, text=_hdr, bg="white", fg="#1c1c1e",
font=_sc_hdr_font, anchor="w",
).grid(row=1, column=_col, sticky="w", padx=8, pady=(4, 0))
_sc_rows = [
("CNPG (PostgreSQL)", "storage.cnpg",
"High-perf SSD — pd-ssd / premium-rwo"),
("Redis / Monitoring", "storage.monitoring",
"Standard HDD — pd-standard / standard-rwo"),
("Garage (S3 store)", "storage.garage",
"Low-cost block — pd-standard / standard-rwo"),
]
self._storage_combos = []
for _r, (_lbl, _key, _note) in enumerate(_sc_rows, start=2):
tk.Label(
storage_tab, text=_lbl, bg="white", fg="#1c1c1e",
font=("SF Pro Text", 10), anchor="w",
).grid(row=_r, column=0, sticky="w", padx=8, pady=3)
_sc_combo = ttk.Combobox(
storage_tab,
textvariable=self.prod_form_vars[_key],
width=28, state="normal",
)
_sc_combo.grid(row=_r, column=1, sticky="w", padx=4, pady=3)
self._storage_combos.append(_sc_combo)
tk.Label(
storage_tab, text=_note, bg="white", fg="#666",
font=("SF Pro Text", 9), anchor="w",
).grid(row=_r, column=2, sticky="w", padx=8, pady=3)
_sc_btn_row = tk.Frame(storage_tab, bg="white")
_sc_btn_row.grid(
row=len(_sc_rows) + 2, column=0, columnspan=3,
sticky="w", padx=8, pady=(8, 2)
)
self._storage_status_var = tk.StringVar(value="")
tk.Button(
_sc_btn_row, text="Fetch from Cluster",
command=self._fetch_storage_classes,
).grid(row=0, column=0, padx=(0, 6))
tk.Label(
_sc_btn_row,
textvariable=self._storage_status_var,
bg="white", fg="#1558b0",
font=("SF Pro Text", 9),
).grid(row=0, column=1, sticky="w")
storage_tab.columnconfigure(2, weight=1)
# ── End GCP Storage mapping tab ─────────────────────────────────────
btn_row = tk.Frame(plan_tab, bg="white")
btn_row.grid(row=0, column=0, columnspan=2, sticky="w", padx=8, pady=4)
save_btn = tk.Button(
btn_row,
text="Save Config",
command=lambda: self._prod_put_config(show_dialog=True),
)
plan_btn = tk.Button(btn_row, text="Plan", command=self._on_prod_plan)
apply_cfg_btn = tk.Button(btn_row, text="Apply", command=self._on_prod_apply)
refresh_btn = tk.Button(
btn_row, text="Refresh Status", command=self._on_prod_refresh_status
)
for idx, btn in enumerate([save_btn, plan_btn, apply_cfg_btn, refresh_btn]):
btn.grid(row=0, column=idx, sticky="w", padx=(0, 6))
tk.Label(
plan_tab,
textvariable=self.prod_apply_status,
bg="white",
fg="#1c1c1e",
anchor="w",
font=("SF Pro Text", 10, "bold"),
).grid(row=1, column=0, columnspan=2, sticky="ew", padx=8, pady=2)
preview_text = tk.Text(
plan_tab,
height=8,
width=95,
bg="#f8f8f8",
fg="black",
wrap="word",
font=("SF Mono", 9),
)
preview_text.grid(row=2, column=0, columnspan=2, sticky="nsew", padx=8, pady=4)
tk.Label(
plan_tab,
text="Apply Progress Logs",
bg="white",
fg="black",
anchor="w",
font=("SF Pro Text", 10, "bold"),
).grid(row=3, column=0, columnspan=2, sticky="w", padx=8, pady=(0, 2))
logs_text = tk.Text(
plan_tab,
height=4,
width=95,
bg="#111111",
fg="#9de79f",
wrap="word",
font=("SF Mono", 9),
)
logs_text.grid(row=4, column=0, columnspan=2, sticky="nsew", padx=8, pady=2)
plan_tab.rowconfigure(2, weight=1)
self._prod_preview_text_widget = preview_text
self._prod_logs_text_widget = logs_text
notebook_win = self.bg_canvas.create_window(
x_label + 20,
y,
window=notebook,
anchor="nw",
width=890,
height=290,
)
self._canvas_items.append(notebook_win)
self._overlay_widgets.append(notebook)
y += 304
self._prod_push_preview_widgets()
if not self.prod_apply_status.get().strip():
self.prod_apply_status.set("idle/ready: Waiting for plan")
# Artifact Registry traffic light — persisted across screens
ar_indicator = self.bg_canvas.create_oval(
x_label + 20, y, x_label + 34, y + 14,
fill="#ff9f0a", outline="#8e8e93", width=1,
)
self._canvas_items.append(ar_indicator)
self._artifact_registry_indicator = ar_indicator
ar_lbl = ui.canvas_text(
self, x_label + 42, y + 7,
"Artifact Registry: checking…",
fill="black", font=("SF Pro Text", 10), anchor="w",
)
self._canvas_items.append(ar_lbl)
self._artifact_registry_label = ar_lbl
y += 28 # noqa: F841 (y consumed; keep for future rows)
self.safe_after(self._check_artifact_registry_async, delay=1000)
ui.canvas_text(
self,
x_label,
y,
"Common Services are configured on the next screen.",
fill="#6e6e73",
font=("SF Pro Text", 11),
)
def _render_common_services_page(self):
# Letterhead at top right
content_width = self.bg_canvas.winfo_width() or 975
right_margin = content_width - 48
ui.canvas_text(
self,
right_margin,
40,
"knoe.dev",
fill="#6e6e73",
font=("SF Pro Text", 32, "bold"),
anchor="ne",
)
ui.canvas_text(
self,
right_margin,
85,
"infrastructure.auto()",
fill="#6e6e73",
font=("SF Pro Text", 18),
anchor="ne",
)
self._render_title("Common Services", y=110)
self._render_paragraph(
"Evaluate, deploy, or repair common services."
" All service traffic lights must be green before continuing.",
y=155,
)
selected_env_key = self._cluster_env_key()
has_cluster = True
if selected_env_key == "dev":
has_cluster = bool((self.selected_k3d_cluster.get() or "").strip())
x_label = 48
y = 220
if not has_cluster:
ui.canvas_text(
self,
x_label,
y,
"Select or create a Dev cluster before configuring common services.",
fill="#6e6e73",
font=("SF Pro Text", 12),
)
self._common_services_success = False
self._service_traffic_lights = {}
self.update_footer()
return
ui.canvas_text(
self,
x_label,
y,
"Common Core Services Namespace:",
fill="black",
font=("SF Pro Text", 12, "bold"),
)
ns_entry = tk.Entry(
self.bg_canvas,
textvariable=self.service_namespace,
width=20,
bg="white",
fg="black",
insertbackground="black",
highlightbackground="#CCCCCC",
highlightthickness=1,
relief="flat",
font=("SF Pro Text", 11),
)
ns_win = self.bg_canvas.create_window(
x_label + 270, y - 6, window=ns_entry, anchor="nw"
)
self._canvas_items.append(ns_win)
self._overlay_widgets.append(ns_entry)
self._service_namespace_entry = ns_entry
self._service_namespace_entry_canvas_window = ns_win
save_btn = tk.Button(
self.bg_canvas,
text="Save",
command=self._validate_and_save_cluster_config,
bg="#F5F5DC",
fg="black",
activebackground="#E5E5D5",
highlightbackground="#F5F5DC",
highlightthickness=0,
relief="flat",
font=("SF Pro Text", 10),
padx=10,
)
save_win = self.bg_canvas.create_window(
x_label + 480, y - 10, window=save_btn, anchor="nw"
)
self._canvas_items.append(save_win)
self._overlay_widgets.append(save_btn)
eval_btn = tk.Button(
self.bg_canvas,
text="Evaluate",
command=self._verify_k3s_services,
bg="#F5F5DC",
fg="black",
activebackground="#E5E5D5",
highlightbackground="#F5F5DC",
highlightthickness=0,
relief="flat",
font=("SF Pro Text", 10),
padx=10,
)
eval_win = self.bg_canvas.create_window(
x_label + 560, y - 10, window=eval_btn, anchor="nw"
)
self._canvas_items.append(eval_win)
self._overlay_widgets.append(eval_btn)
deploy_btn = tk.Button(
self.bg_canvas,
text="Deploy",
command=self._deploy_k3s_services,
bg="#F5F5DC",
fg="black",
activebackground="#E5E5D5",
highlightbackground="#F5F5DC",
highlightthickness=0,
relief="flat",
font=("SF Pro Text", 10),
padx=10,
)
deploy_win = self.bg_canvas.create_window(
x_label + 660, y - 10, window=deploy_btn, anchor="nw"
)
self._canvas_items.append(deploy_win)
self._overlay_widgets.append(deploy_btn)
self._k3s_deploy_btn = deploy_btn
repair_btn = tk.Button(
self.bg_canvas,
text="Repair",
command=self._deploy_k3s_services,
bg="#F5F5DC",
fg="black",
activebackground="#E5E5D5",
highlightbackground="#F5F5DC",
highlightthickness=0,
relief="flat",
font=("SF Pro Text", 10),
padx=10,
)
repair_win = self.bg_canvas.create_window(
x_label + 758, y - 10, window=repair_btn, anchor="nw"
)
self._canvas_items.append(repair_win)
self._overlay_widgets.append(repair_btn)
y += 42
components = [
("registry", "Registry"),
("certmgr", "Cert Manager"),
("garage", "Garage"),
("kong", "Kong"),
("openbao", "OpenBao"),
("opentofu", "OpenTofu"),
]
lights = {}
for idx, (key, label) in enumerate(components):
row = idx // 3
col = idx % 3
lx = x_label + 20 + (col * 290)
ly = y + 8 + (row * 36)
indicator = self.bg_canvas.create_oval(
lx,
ly,
lx + 14,
ly + 14,
fill="#ff9f0a",
outline="#8e8e93",
width=1,
)
self._canvas_items.append(indicator)
ui.canvas_text(
self,
lx + 22,
ly + 7,
label,
fill="black",
font=("SF Pro Text", 10),
anchor="w",
)
lights[key] = indicator
self._service_traffic_lights = lights
y += 90
notebook = ttk.Notebook(self.bg_canvas)
status_tab = tk.Frame(notebook, bg="white")
deploy_tab = tk.Frame(notebook, bg="white")
notebook.add(status_tab, text="Evaluate")
notebook.add(deploy_tab, text="Deploy / Repair")
status_console = ui.TerminalConsole(status_tab, highlightthickness=0, bd=0)
status_console.pack(fill="both", expand=True, padx=4, pady=4)
deploy_console = ui.TerminalConsole(deploy_tab, highlightthickness=0, bd=0)
deploy_console.pack(fill="both", expand=True, padx=4, pady=4)
notebook_win = self.bg_canvas.create_window(
x_label,
y,
window=notebook,
anchor="nw",
width=890,
height=430,
)
self._canvas_items.append(notebook_win)
self._overlay_widgets.extend(
[notebook, status_tab, deploy_tab, status_console, deploy_console]
)
self._k3s_service_notebook = notebook
self._k3s_service_status_tab = status_tab
self._k3s_service_deploy_tab = deploy_tab
self._k3s_service_status_console = status_console
self._k3s_service_deploy_console = deploy_console
self._common_services_success = bool(
getattr(self, "_common_services_success", False)
or getattr(self, "_all_services_green", False)
)
self.update_footer()
self._verify_k3s_services()
def _cluster_env_key(self, env_label: str | None = None) -> str:
"""Return normalized cluster env key ('dev', 'service', 'prod')."""
value = env_label
if value is None:
try:
value = self.cluster_env.get()
except Exception:
value = ""
try:
if hasattr(value, "get") and callable(value.get):
value = value.get()
except Exception:
pass
if not value:
try:
value = (
(self.knoe_cfg_data.get("Initialize Cluster", {}) or {})
.get("ENVIRONMENT", "")
.strip()
)
except Exception:
value = ""
if not value:
try:
value = (
(self.knoe_cfg_data.get("Global", {}) or {})
.get("CLUSTER_ENV", "")
.strip()
)
except Exception:
value = ""
key = _normalize_cluster_env(value)
if not key:
return "dev"
return key
def _is_valid_k8s_namespace(self, value: str | None) -> bool:
ns = _safe_str(value).strip()
if not ns or len(ns) > 63:
return False
return re.fullmatch(r"[a-z0-9]([-a-z0-9]*[a-z0-9])?", ns) is not None
def _get_service_namespace(self) -> str:
"""Resolve service namespace with UI/env/config fallbacks.
Safety rule: avoid defaulting to the Kubernetes `default` namespace unless we truly
cannot infer a better option.
"""
# 1) Explicit user/config/env choice wins.
candidate_namespaces: list[str] = []
try:
candidate_namespaces.append((self.service_namespace.get() or "").strip())
except Exception:
pass
try:
candidate_namespaces.append(
(
(self.knoe_cfg_data.get("Global", {}) or {})
.get("SERVICE_NAMESPACE", "")
.strip()
)
)
except Exception:
pass
candidate_namespaces.append((os.environ.get("SERVICE_NAMESPACE") or "").strip())
for ns in candidate_namespaces:
if self._is_valid_k8s_namespace(ns):
return ns
# 2) If the cluster is reachable, infer the common services namespace.
inferred = ""
try:
inferred = (self._infer_common_services_namespace() or "").strip()
except Exception:
inferred = ""
if self._is_valid_k8s_namespace(inferred):
return inferred
# 3) Stable defaults for known environments.
try:
# All managed environments (dev/k3d, service/k3s, prod/k8s) use knoe-system.
if self._cluster_env_key() in ("dev", "service", "prod"):
return "knoe-system"
except Exception:
pass
# 4) Absolute last resort.
return "default"
def _infer_common_services_namespace(self) -> str:
"""Infer the namespace hosting common services.
If the cluster is reachable, try to locate an existing Docker Registry workload
(image containing `registry:2`). If found, treat that namespace as the common
services namespace.
This method is read-only.
"""
# Cache to avoid repeated `kubectl -A` probes during periodic refresh.
cache_ttl_s = 60
now = time.time()
cached_ns = getattr(self, "_common_services_ns_cache", "")
cached_at = getattr(self, "_common_services_ns_cache_at", 0.0)
if cached_ns and (now - float(cached_at or 0.0)) < cache_ttl_s:
return cached_ns
# Ensure kubeconfig is available when possible (read-only action).
try:
self._ensure_k3s_kubeconfig_merged()
except Exception:
pass
# Build a kubectl command suitable for the active mode.
try:
mode = self._deployment_mode()
except Exception:
mode = None
try:
base_cmd = self._kubectl_base_cmd(mode)
except Exception:
base_cmd = ["kubectl"]
# Respect an existing kubeconfig if present.
env = os.environ.copy()
try:
kc = _find_kubeconfig_file(env)
except Exception:
kc = _find_kubeconfig_file()
if kc:
env["KUBECONFIG"] = kc
def _probe(kind: str) -> list[str]:
cmd = base_cmd + ["get", kind, "-A", "-o", "json"]
res = subprocess.run(
cmd,
capture_output=True,
text=True,
env=env,
timeout=5,
)
if res.returncode != 0:
return []
try:
data = json.loads(res.stdout or "{}")
except Exception:
return []
out = []
for item in (data.get("items") or []):
try:
ns = (item.get("metadata") or {}).get("namespace") or ""
containers = (
(((item.get("spec") or {}).get("template") or {}).get("spec") or {}).get(
"containers"
)
or []
)
for c in containers:
img = (c.get("image") or "").strip()
if "registry:2" in img:
if ns:
out.append(ns)
break
except Exception:
continue
return out
namespaces = []
# Deployments are the most common shape; fall back to StatefulSets.
namespaces.extend(_probe("deploy"))
if not namespaces:
namespaces.extend(_probe("sts"))
# Prefer knoe-system if present, otherwise prefer a non-default namespace.
ns_choice = ""
if namespaces:
if "knoe-system" in namespaces:
ns_choice = "knoe-system"
else:
non_default = [n for n in namespaces if n and n != "default"]
ns_choice = (non_default[0] if non_default else namespaces[0]) or ""
if ns_choice:
setattr(self, "_common_services_ns_cache", ns_choice)
setattr(self, "_common_services_ns_cache_at", now)
return ns_choice
def _on_cluster_env_change(self, *args):
# Keep configuration environment selection explicit and stable.
try:
conf_dir = self._resolve_knoe_conf_dir()
knoe_conf.activate_environment(conf_dir, self.cluster_env.get())
except Exception:
pass
self._set_deploy_target_from_cluster_env()
# Trigger status check or refresh UI
env_key = self._cluster_env_key()
if env_key == "service":
try:
self._apply_k3s_defaults()
except Exception:
pass
# Prefer knoe-k3s context if present, but require explicit Apply.
try:
values = self._get_kubectx_list()
except Exception:
values = []
if hasattr(self, "selected_kubectx") and "knoe-k3s" in (values or []):
self.selected_kubectx.set("knoe-k3s")
self._update_kubectx_apply_button()
elif env_key == "dev":
name = self.selected_k3d_cluster.get()
if name:
if hasattr(self, "selected_kubectx"):
self.selected_kubectx.set(f"k3d-{name}")
self._switch_kubectx(f"k3d-{name}")
self._verify_k3s_services()
self.show_page("init_cluster")
def _deployment_mode(self) -> str:
try:
env_val = self.cluster_env.get()
except Exception:
env_val = ""
return _deployment_mode_from_env(env_val)
def _deployment_pipeline_url(self, target_key: str) -> str:
env_url = (
os.environ.get("PROLE_OPENTOFU_URL") or os.environ.get("OPENTOFU_URL") or ""
).strip()
if env_url:
return env_url
section = None
if target_key == "service":
section = "Service Cluster (k3s)"
elif target_key == "prod":
section = "Prod Cluster (k8s)"
if section:
url = (
(self.knoe_cfg_data.get(section, {}) or {})
.get("PIPELINE_URL", "")
.strip()
)
if url:
return url
return _default_opentofu_pipeline_url()
def _set_deploy_target_from_cluster_env(self):
if not hasattr(self, "deploy_target"):
return
if getattr(self, "_syncing_deploy_target", False):
return
self._syncing_deploy_target = True
try:
self.deploy_target.set(_deployment_target_label(self.cluster_env.get()))
except Exception:
pass
finally:
self._syncing_deploy_target = False
def _on_deploy_target_change(self, *args):
if getattr(self, "_syncing_deploy_target", False):
return
target = ""
try:
target = self.deploy_target.get()
except Exception:
target = ""
key = _normalize_cluster_env(target)
if key == "dev":
desired = "dev"
elif key == "service":
desired = "service"
elif key == "prod":
desired = "prod"
else:
return
self._syncing_deploy_target = True
try:
if self.cluster_env.get() != desired:
self.cluster_env.set(desired)
finally:
self._syncing_deploy_target = False
def _dashboard_kong_status(self, env: str | None = None) -> dict:
"""Check Kubernetes Dashboard (Kong) pod health and return status info."""
ns = "kubernetes-dashboard"
base_cmd = self._kubectl_base_cmd(env)
def _pod_healthy(pod_status: dict) -> bool:
phase = (pod_status.get("phase") or "").strip()
if phase not in ("Running", "Succeeded"):
return False
statuses = pod_status.get("containerStatuses") or []
if not statuses:
return phase == "Succeeded"
for cs in statuses:
if cs.get("ready") is True:
continue
state = cs.get("state") or {}
waiting = state.get("waiting") or {}
reason = (waiting.get("reason") or "").lower()
if reason:
return False
return False
return True
# Ensure namespace exists
rc_ns, out_ns = self._run_cmd_capture(
base_cmd + ["get", "ns", ns, "-o", "name"], timeout=6
)
if rc_ns != 0 or ns not in (out_ns or ""):
return {
"ok": False,
"msg": "Dashboard (Kong): Missing",
"fill": "#ff9f0a",
"reset_pods": [],
}
rc, out = self._run_cmd_capture(
base_cmd + ["get", "pods", "-n", ns, "-o", "json"], timeout=8
)
if rc != 0:
return {
"ok": False,
"msg": "Dashboard (Kong): Unable to query",
"fill": "#ff9f0a",
"reset_pods": [],
}
try:
payload = json.loads(out or "{}")
except Exception:
return {
"ok": False,
"msg": "Dashboard (Kong): Unable to parse status",
"fill": "#ff9f0a",
"reset_pods": [],
}
kong_pods = []
for item in payload.get("items", []):
name = (item.get("metadata", {}) or {}).get("name", "")
name_lc = name.lower()
if "kong" in name_lc and "dashboard" in name_lc:
kong_pods.append(item)
if not kong_pods:
return {
"ok": False,
"msg": "Dashboard (Kong): Missing",
"fill": "#ff9f0a",
"reset_pods": [],
}
unhealthy = []
for item in kong_pods:
name = (item.get("metadata", {}) or {}).get("name", "")
status = item.get("status") or {}
if not _pod_healthy(status):
unhealthy.append(name)
if unhealthy:
return {
"ok": False,
"msg": "Dashboard (Kong): Unhealthy (safe to reset)",
"fill": "#ff9f0a",
"reset_pods": unhealthy,
}
return {
"ok": True,
"msg": "Dashboard (Kong): Running",
"fill": "#34c759",
"reset_pods": [],
}
def _set_cluster_env_message(
self, text: str, color: str = "#34c759", clear_after_ms: int | None = None
):
"""Set a transient status message on the Cluster Environment screen."""
def _apply():
if not hasattr(self, "_cluster_env_message_label"):
return
if not self.bg_canvas.winfo_exists():
return
try:
self.bg_canvas.itemconfig(
self._cluster_env_message_label, text=text, fill=color
)
except Exception:
pass
self.safe_after(_apply)
if clear_after_ms is not None and clear_after_ms >= 0:
def _clear():
if not hasattr(self, "_cluster_env_message_label"):
return
if not self.bg_canvas.winfo_exists():
return
try:
self.bg_canvas.itemconfig(self._cluster_env_message_label, text="")
except Exception:
pass
self.safe_after(_clear, delay=clear_after_ms)
def _authority_context_missing(self) -> bool:
"""Detect missing authority Docker context when Kerberos is enabled."""
try:
if not self.kerberos_enabled.get():
return False
except Exception:
return False
candidates = []
env_home = (os.environ.get("KNOE_HOME") or "").strip()
if env_home:
candidates.append(Path(env_home))
candidates.append(PROJECT_ROOT)
for base in candidates:
try:
if (base / "authority").is_dir():
return False
if (base / "knoe" / "authority").is_dir():
return False
except Exception:
continue
return True
def _maybe_run_repair_pipeline(self, status: dict, kong_status: dict | None):
"""Run repair pipeline when cluster is ready and anomalies are detected."""
if not status.get("cluster_ok"):
return
anomalies = []
if kong_status and not kong_status.get("ok", True):
anomalies.append("dashboard")
if self._authority_context_missing():
anomalies.append("authority")
if not anomalies:
return
now = time.time()
if self._repair_inflight:
return
if now - self._repair_last_run_at < self._repair_cooldown_s:
return
self._repair_last_run_at = now
self._repair_inflight = True
def worker():
try:
env_key = status.get("env") or self._cluster_env_key()
if env_key == "service":
ns = self._get_service_namespace()
else:
ns = (self.db_namespace.get() or "").strip() or "default"
env = self._script_env_for_namespace(ns)
if env_key == "service":
env["KNOE_MODE"] = "k3s"
elif env_key == "dev":
env["KNOE_MODE"] = "k3d"
self._set_cluster_env_message(
"Repair pipeline started", "#34c759", clear_after_ms=3000
)
self.controller.run_script(
"repair_pipeline.sh",
args=["-n", ns],
env=env,
)
self._set_cluster_env_message(
"Repair pipeline complete", "#34c759", clear_after_ms=3000
)
except Exception as e:
self._set_cluster_env_message(
f"Repair pipeline failed: {e}", "#ff3b30", clear_after_ms=6000
)
finally:
self._repair_inflight = False
threading.Thread(target=worker, daemon=True).start()
def _check_opentofu_health(self) -> bool:
"""Check OpenTofu readiness in the selected namespace."""
try:
env_key = self._cluster_env_key()
except Exception:
env_key = "dev"
try:
if env_key == "service":
ns = self._get_service_namespace()
else:
ns = (self.db_namespace.get() or "").strip() or "default"
except Exception:
ns = "default"
try:
base_cmd = self._kubectl_base_cmd(env_key)
res_dep = subprocess.run(
base_cmd
+ [
"-n",
ns,
"get",
"deploy",
"opentofu",
"-o",
"jsonpath={.status.readyReplicas}",
],
capture_output=True,
text=True,
timeout=3,
)
ready_dep = res_dep.returncode == 0 and (
res_dep.stdout or "0"
).strip() not in ("", "0")
if ready_dep:
return True
res_sts = subprocess.run(
base_cmd
+ [
"-n",
ns,
"get",
"statefulset",
"opentofu",
"-o",
"jsonpath={.status.readyReplicas}",
],
capture_output=True,
text=True,
timeout=3,
)
ready_sts = res_sts.returncode == 0 and (
res_sts.stdout or "0"
).strip() not in ("", "0")
return ready_sts
except Exception:
return False
def _check_k8s_cluster(self, env_label: str) -> tuple[bool, str]:
env_key = self._cluster_env_key(env_label)
label = (
"K3s Cluster"
if env_key == "service"
else "Prod Cluster" if env_key == "prod" else "Kubernetes Cluster"
)
try:
kubectl = subprocess.run(["which", "kubectl"], capture_output=True)
if kubectl.returncode != 0:
return False, f"{label}: kubectl not found"
if env_key == "service":
kubeconfig = _find_kubeconfig_file()
if not kubeconfig:
server, token = self._k3s_connection_info()
if not server:
return False, f"{label}: missing server URL or kubeconfig"
cmd = self._kubectl_base_cmd(env_key) + ["cluster-info"]
res = subprocess.run(cmd, capture_output=True, text=True, timeout=8)
ok = res.returncode == 0
except Exception:
ok = False
msg = f"{label}: Connected" if ok else f"{label}: Not reachable"
return ok, msg
def _get_k3d_cluster_list(self) -> list[str]:
"""Get list of k3d clusters."""
try:
res = subprocess.run(
["k3d", "cluster", "list", "--no-headers"],
capture_output=True,
text=True,
timeout=5,
)
if res.returncode == 0:
# Extract first column (cluster names)
return [
line.split()[0] for line in res.stdout.splitlines() if line.strip()
]
except Exception:
pass
return []
def _on_create_k3d_cluster(self):
name = self.selected_k3d_cluster.get().strip()
if not name:
messagebox.showerror("Error", "Cluster name cannot be empty")
return
# Persist the typed name so the refreshed page keeps it selected
self.k3d_cluster_name.set(name)
# If the cluster already exists just select it (triggers notebook display)
existing = self._get_k3d_cluster_list()
if name in existing:
self.selected_k3d_cluster.set(name)
self.show_page("init_cluster")
return
# Refresh the page so the notebook/console is visible before streaming output
self.selected_k3d_cluster.set(name)
self.show_page("init_cluster")
def do_create():
console = getattr(self, "_k3s_service_deploy_console", None)
if console:
console.clear()
console.write(
f"Creating k3d cluster '{name}' @ {time.strftime('%Y-%m-%d %H:%M:%S')}\n\n"
)
try:
if self._k3s_service_notebook and self._k3s_service_deploy_tab:
self._k3s_service_notebook.select(self._k3s_service_deploy_tab)
except Exception:
pass
try:
# Build the k3d create command
knoe_data = str(self._resolve_env_dir("PROLE_DATA", "data"))
volume_args = _k3d_knoe_data_volume_args(knoe_data)
reg_args = []
if getattr(self, "registry_url", None):
if self.registry_url.startswith("localhost:5000"):
reg_args = ["--registry-create", "knoe-registry:0.0.0.0:5000"]
else:
reg_args = ["--registry-use", self.registry_url]
cmd = (
["k3d", "cluster", "create", name, "-a", "2", "--wait"]
+ volume_args
+ reg_args
+ ["--timestamps"]
)
if console:
console.write(f"$ {' '.join(cmd)}\n\n")
proc = subprocess.Popen(
cmd,
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT,
text=True,
cwd=PROJECT_ROOT,
)
for line in proc.stdout:
if console:
console.write(line)
proc.wait()
if proc.returncode != 0:
if console:
console.write(
f"\nk3d cluster create failed (exit {proc.returncode})\n"
)
return
if console:
console.write(f"\nCluster '{name}' created successfully.\n\n")
# Merge kubeconfig so kubectl can reach the new cluster
try:
subprocess.run(
[
"k3d",
"kubeconfig",
"merge",
name,
"--kubeconfig-switch-context",
],
capture_output=True,
text=True,
timeout=10,
)
if console:
console.write(f"Kubeconfig merged for cluster '{name}'.\n\n")
except Exception as kce:
if console:
console.write(f"Warning: failed to merge kubeconfig: {kce}\n")
# Refresh UI and then deploy common services check
self.safe_after(self._refresh_k3d_ui)
# Trigger common services check in the deploy console
self.safe_after(self._deploy_k3s_services)
except Exception as e:
if console:
console.write(f"\nError: {e}\n")
self.safe_after(
lambda: messagebox.showerror(
"Error", f"Failed to create k3d cluster: {e}"
)
)
threading.Thread(target=do_create, daemon=True).start()
def _refresh_k3d_ui(self):
clusters = self._get_k3d_cluster_list()
self.k3d_cluster_list.set(clusters)
name = self.k3d_cluster_name.get().strip()
if name in clusters:
self.selected_k3d_cluster.set(name)
self.show_page("init_cluster")
def _on_k3d_cluster_select(self, *args):
"""Called when a k3d cluster is selected from the dropdown."""
cluster_name = self.selected_k3d_cluster.get()
if cluster_name:
try:
# Merge kubeconfig and switch context to the selected cluster
subprocess.run(
["k3d", "kubeconfig", "merge", cluster_name, "--switch-context"],
capture_output=True,
text=True,
)
# Ensure kubectx also knows about it
self._switch_kubectx(f"k3d-{cluster_name}")
except Exception:
pass
self.show_page("init_cluster")
def _on_delete_k3d_cluster(self):
"""Delete the selected k3d cluster after user confirmation."""
name = self.selected_k3d_cluster.get().strip()
if not name:
messagebox.showerror("Error", "No cluster selected to delete.")
return
confirmed = messagebox.askyesno(
"Confirm Delete", f"Are you sure you want to delete cluster '{name}'?"
)
if not confirmed:
return
# Show the notebook/console before streaming output
self.show_page("init_cluster")
def do_delete():
console = getattr(self, "_k3s_service_deploy_console", None)
if console:
console.clear()
console.write(
f"Deleting k3d cluster '{name}' @ {time.strftime('%Y-%m-%d %H:%M:%S')}\n\n"
)
try:
if self._k3s_service_notebook and self._k3s_service_deploy_tab:
self._k3s_service_notebook.select(self._k3s_service_deploy_tab)
except Exception:
pass
try:
cmd = ["k3d", "cluster", "delete", name]
if console:
console.write(f"$ {' '.join(cmd)}\n\n")
proc = subprocess.Popen(
cmd,
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT,
text=True,
cwd=PROJECT_ROOT,
)
for line in proc.stdout:
if console:
console.write(line)
proc.wait()
if proc.returncode != 0:
if console:
console.write(
f"\nk3d cluster delete failed (exit {proc.returncode})\n"
)
return
if console:
console.write(f"\nCluster '{name}' deleted successfully.\n")
# Clear selection and refresh UI
self.selected_k3d_cluster.set("")
self.k3d_cluster_name.set("")
self.safe_after(self._refresh_k3d_ui)
except Exception as e:
if console:
console.write(f"\nError: {e}\n")
self.safe_after(
lambda: messagebox.showerror(
"Error", f"Failed to delete k3d cluster: {e}"
)
)
threading.Thread(target=do_delete, daemon=True).start()
def _on_kubectx_select(self, *args):
"""Called when a generic kubernetes context is selected from the dropdown."""
self._update_kubectx_apply_button()
ctx = (self.selected_kubectx.get() or "").strip()
if ctx:
self._set_cluster_env_message(
f"Context change pending: {ctx} (click Apply)",
"#ff9500",
clear_after_ms=4000,
)
def _on_kubectx_apply(self):
ctx = (self.selected_kubectx.get() or "").strip()
if not ctx:
return
ok = self._switch_kubectx(ctx)
if ok:
self._kubectx_applied = ctx
self._update_kubectx_apply_button()
self._set_cluster_env_message(
f"Context applied: {ctx}", "#34c759", clear_after_ms=3000
)
self._verify_k3s_services()
self.show_page("init_cluster")
def _on_dual_kubectx_select(self, *args):
"""Called when App or DB cluster context is selected in prod/k8s mode."""
app_ctx = (self.app_cluster_kubecontext.get() or "").strip()
db_ctx = (self.db_cluster_kubecontext.get() or "").strip()
if app_ctx and db_ctx:
self._set_cluster_env_message(
f"App: {app_ctx} / DB: {db_ctx} — click Apply to save",
"#ff9500",
clear_after_ms=5000,
)
def _on_kubectx_apply_dual(self):
"""Save explicit App + DB cluster contexts to knoe.cfg [Inputs] and [Global]."""
app_ctx = (self.app_cluster_kubecontext.get() or "").strip()
db_ctx = (self.db_cluster_kubecontext.get() or "").strip()
if not app_ctx or not db_ctx:
self._set_cluster_env_message(
"Both App Cluster and DB Cluster must be selected.", "#ff3b30"
)
return
# Persist to [Inputs] so they survive installer regeneration.
inputs = self.knoe_cfg_data.setdefault("Inputs", {})
inputs["init_cluster.app_cluster_kubecontext"] = app_ctx
inputs["init_cluster.db_cluster_kubecontext"] = db_ctx
inputs["env_setup.APP_CLUSTER_KUBECONTEXT"] = app_ctx
inputs["env_setup.DB_CLUSTER_KUBECONTEXT"] = db_ctx
# Global.KUBECONTEXT = APP cluster — common services (garage, openbao, kong,
# monitoring) live there. DB cluster is always accessed via DB_CLUSTER_KUBECONTEXT.
glob = self.knoe_cfg_data.setdefault("Global", {})
glob["KUBECONTEXT"] = app_ctx
glob["APP_CLUSTER_KUBECONTEXT"] = app_ctx
glob["DB_CLUSTER_KUBECONTEXT"] = db_ctx
self._save_knoe_cfg()
# Switch ambient kubectl context to the app cluster (matches Global.KUBECONTEXT).
self._switch_kubectx(app_ctx)
self._set_cluster_env_message(
f"Saved — App: {app_ctx} / DB: {db_ctx}", "#34c759", clear_after_ms=4000
)
self.show_page("init_cluster")
def _update_kubectx_apply_button(self):
btn = getattr(self, "_kubectx_apply_btn", None)
if not btn:
return
# Prod (dual-context) mode: enable only when both contexts are non-empty.
env_key = ""
try:
env_key = (getattr(self, "cluster_env", None) and self.cluster_env.get() or "").strip()
except Exception:
pass
if env_key == "prod":
app_ctx = ""
db_ctx = ""
try:
app_ctx = (self.app_cluster_kubecontext.get() or "").strip()
except Exception:
pass
try:
db_ctx = (self.db_cluster_kubecontext.get() or "").strip()
except Exception:
pass
state = "normal" if (app_ctx and db_ctx) else "disabled"
else:
selected = ""
try:
if hasattr(self, "selected_kubectx") and self.selected_kubectx is not None:
selected = (self.selected_kubectx.get() or "").strip()
except Exception:
selected = ""
applied = (getattr(self, "_kubectx_applied", "") or "").strip()
state = "normal" if (selected and selected != applied) else "disabled"
try:
btn.config(state=state)
except Exception:
pass
def _switch_kubectx(self, context_name: str) -> bool:
"""Switch the current kubernetes context using kubectx or kubectl."""
if not context_name:
return False
# Determine which kubeconfig to modify.
# Use the same logic as the rest of the application.
kc = _find_kubeconfig_file()
env = os.environ.copy()
if kc:
env["KUBECONFIG"] = kc
try:
# Try kubectx first
res = subprocess.run(
["kubectx", context_name], capture_output=True, text=True, env=env
)
ok = res.returncode == 0
err = (res.stderr or "").strip()
if not ok:
# Fallback to kubectl config use-context
res2 = subprocess.run(
["kubectl", "config", "use-context", context_name],
capture_output=True,
text=True,
env=env
)
ok = res2.returncode == 0
if not ok:
err2 = (res2.stderr or "").strip()
err = err2 or err
# Update the UI dropdown to match (but don't trigger re-select)
if (
ok
and hasattr(self, "selected_kubectx")
and self.selected_kubectx.get() != context_name
):
try:
# Check if it's in the current values list
current_values = []
if hasattr(self, "kubectx_list") and self.kubectx_list is not None:
current_values = list(self.kubectx_list.get() or [])
if not current_values:
current_values = self._get_kubectx_list()
if context_name in (current_values or []):
self.selected_kubectx.set(context_name)
except Exception:
pass
if not ok:
try:
messagebox.showerror(
"Kubernetes Context",
f"Failed to switch to context '{context_name}'.\n\n{err or 'Unknown error'}",
)
except Exception:
pass
return ok
except Exception as e:
try:
messagebox.showerror(
"Kubernetes Context",
f"Failed to switch to context '{context_name}': {e}",
)
except Exception:
pass
return False
def _current_kubectl_context(self) -> str:
kc = _find_kubeconfig_file()
env = os.environ.copy()
if kc:
env["KUBECONFIG"] = kc
try:
res = subprocess.run(
["kubectl", "config", "current-context"],
capture_output=True,
text=True,
env=env,
)
if res.returncode == 0:
return (res.stdout or "").strip()
except Exception:
pass
return ""
def _get_kubectx_list(self) -> list[str]:
"""Get list of kubernetes contexts."""
kc = _find_kubeconfig_file()
env = os.environ.copy()
if kc:
env["KUBECONFIG"] = kc
try:
res = subprocess.run(["kubectx"], capture_output=True, text=True, env=env)
if res.returncode == 0:
return res.stdout.strip().split("\n")
# Fallback to kubectl
res = subprocess.run(
["kubectl", "config", "get-contexts", "-o", "name"],
capture_output=True,
text=True,
env=env
)
if res.returncode == 0:
return res.stdout.strip().split("\n")
except Exception:
pass
return ["default"]
def _ensure_prod_config_state(self):
if getattr(self, "_prod_config_state_ready", False):
return
self._prod_config_state_ready = True
self.prod_config_api = getattr(self, "prod_config_api", ProdConfigApi())
self.prod_form_vars = {
"metadata.name": tk.StringVar(value="knoey-root"),
"metadata.environment": tk.StringVar(value="production"),
"cloud.provider": tk.StringVar(value="gcp"),
"cloud.projectId": tk.StringVar(value=""),
"cloud.region": tk.StringVar(value="us-central1"),
"cloud.clusterName": tk.StringVar(value="knoe-prod"),
"cloud.vpcMode": tk.StringVar(value="managed"),
"cloud.vpcName": tk.StringVar(value=""),
"cloud.subnetName": tk.StringVar(value=""),
"cloud.artifactRegistry": tk.StringVar(value=""),
"cloud.dnsZone": tk.StringVar(value="knoe-dev-zone"),
"kubernetes.namespace": tk.StringVar(value="ecosystem-0"),
"database.clusterName": tk.StringVar(value="knoe-db"),
"database.postgresVersion": tk.StringVar(value="16"),
"database.instances": tk.StringVar(value="3"),
"database.storageClass": tk.StringVar(value="premium-rwo"),
"database.storageSizeGi": tk.StringVar(value="100"),
"database.appDatabase": tk.StringVar(value="knoey"),
"database.metaDatabase": tk.StringVar(value="knoe_meta"),
"database.appUser": tk.StringVar(value="knoey_app"),
"database.adminUser": tk.StringVar(value="knoe_admin"),
"backups.backupBucket": tk.StringVar(value="knoe-0-backups"),
"backups.walBucket": tk.StringVar(value="knoe-0-wal"),
"backups.retentionDays": tk.StringVar(value="14"),
"auth.provider": tk.StringVar(value="google-workspace-oidc"),
"auth.issuer": tk.StringVar(value="https://accounts.google.com"),
"auth.clientId": tk.StringVar(value="secretref://google-oidc-client-id"),
"auth.clientSecret": tk.StringVar(
value="secretref://google-oidc-client-secret"
),
"auth.bootstrapAdminEmail": tk.StringVar(value="admin@knoey.com"),
"routing.frontdoorHost": tk.StringVar(value="knoey.com"),
"routing.platformDomain": tk.StringVar(value="knoe.dev"),
"routing.tlsMode": tk.StringVar(value="managed"),
"migration.sourceEnvironment": tk.StringVar(value="knoe.org"),
"migration.mode": tk.StringVar(value="snapshot-restore"),
"migration.sourceHost": tk.StringVar(value="knoe-local-db.knoe.org"),
"migration.sourcePort": tk.StringVar(value="5432"),
"migration.sourceDatabase": tk.StringVar(value="knoey"),
"migration.sourceUser": tk.StringVar(value="replication_user"),
"migration.sourcePasswordRef": tk.StringVar(
value="secretref://local-source-db-password"
),
# GKE node pool configuration (persisted to [Initialize Cluster] in knoe.cfg)
"init_cluster.db_boot_disk_type": tk.StringVar(value="pd-standard"),
"init_cluster.db_boot_disk_size_gb": tk.StringVar(value="50"),
}
# Pre-populate GKE node pool vars from existing knoe.cfg if present
_init_clus = (getattr(self, "knoe_cfg_data", None) or {}).get("Initialize Cluster") or {}
if _init_clus.get("DB_BOOT_DISK_TYPE"):
self.prod_form_vars["init_cluster.db_boot_disk_type"].set(_init_clus["DB_BOOT_DISK_TYPE"])
if _init_clus.get("DB_BOOT_DISK_SIZE_GB"):
self.prod_form_vars["init_cluster.db_boot_disk_size_gb"].set(_init_clus["DB_BOOT_DISK_SIZE_GB"])
self.prod_bool_vars = {
"migration.continuousUntilCutover": tk.BooleanVar(value=False)
}
self.prod_tab_messages = tk.StringVar(value="")
self.prod_apply_status = tk.StringVar(value="")
self.prod_preview_yaml = ""
self.prod_preview_tf_vars = ""
self.prod_preview_install_plan = ""
self.prod_apply_logs = ""
self._prod_preview_text_widget = None
self._prod_logs_text_widget = None
# Scan conf/ for all gcp.cfg files; prefer conf/prod/gcp.cfg
if not getattr(self, "_gcp_cfg_values", None):
try:
import glob as _glob
conf_dir = self._resolve_knoe_conf_dir()
found = sorted(_glob.glob(str(conf_dir / "**" / "gcp.cfg"), recursive=True))
self._gcp_cfg_files = found
prod_path = conf_dir / "prod" / "gcp.cfg"
chosen = prod_path if prod_path.exists() else (Path(found[0]) if found else None)
if chosen:
loaded = _parse_gcp_cfg(chosen)
if loaded:
upper = {k.upper(): v for k, v in loaded.items()}
self.knoe_cfg_data.setdefault("GCP", {}).update(upper)
self._gcp_cfg_values = dict(upper)
except Exception:
pass
try:
saved_yaml = (
(getattr(self, "knoe_cfg_data", None) or {})
.get("Prod Cluster (k8s)", {})
.get("PRODUCTION_CONFIG_YAML", "")
.strip()
)
if saved_yaml:
import yaml as _yaml
saved_doc = _yaml.safe_load(saved_yaml) or {}
restored = build_config(saved_doc)
self.prod_config_api.put_prod_config(to_api_payload(restored))
payload = self.prod_config_api.get_prod_config()
self._prod_apply_payload_to_vars(payload)
except Exception:
pass
# Apply gcp.cfg values AFTER payload restore — only fill fields that are truly empty.
# Never overwrite a value the user set (e.g. us-west3) with a default or assumed value.
gcp = getattr(self, "_gcp_cfg_values", {}) or {}
if gcp.get("PROJECT_ID") and not self.prod_form_vars["cloud.projectId"].get().strip():
self.prod_form_vars["cloud.projectId"].set(gcp["PROJECT_ID"])
if gcp.get("REGION") and not self.prod_form_vars["cloud.region"].get().strip():
self.prod_form_vars["cloud.region"].set(gcp["REGION"])
def _fetch_gke_clusters(self):
"""Fetch GKE clusters for the current project in a background thread and populate the cluster Listbox."""
import sys as _sys
listbox = getattr(self, "_gke_cluster_listbox", None)
status_var = getattr(self, "_gke_cluster_status_var", None)
if listbox is None:
return
if getattr(self, "_gcloud_auth_ok", None) is False:
if status_var:
status_var.set("⚠ gcloud not authenticated — use Refresh Auth")
return
if status_var:
status_var.set("⏳ loading clusters…")
_project_id = self.prod_form_vars.get("cloud.projectId")
_project_id = _project_id.get().strip() if _project_id is not None else ""
def _worker():
clusters: list[dict] = []
error = ""
auth_error = False
project_id = _project_id
cmd = [
"gcloud", "container", "clusters", "list",
"--format=value(name,location,status,currentMasterVersion)",
]
if project_id:
cmd += ["--project", project_id]
try:
result = subprocess.run(
cmd,
capture_output=True, text=True, timeout=30,
)
if result.returncode == 0:
for line in result.stdout.splitlines():
parts = line.split("\t")
if len(parts) >= 2:
clusters.append({
"name": parts[0].strip(),
"location": parts[1].strip(),
"status": parts[2].strip() if len(parts) > 2 else "",
"version": parts[3].strip() if len(parts) > 3 else "",
})
else:
stderr_text = result.stderr.strip()
if stderr_text:
print(f"[gcloud container clusters list] {stderr_text}", file=_sys.stderr)
first_line = (stderr_text.splitlines() or [""])[0]
auth_keywords = ("reauthentication", "auth token", "not authenticated",
"login", "credential")
if any(kw in first_line.lower() for kw in auth_keywords):
auth_error = True
error = "Authentication required"
elif not project_id and "required property [project]" in stderr_text.lower():
error = "project ID not set — enter Project ID above"
else:
error = first_line or "gcloud error"
except FileNotFoundError:
error = "gcloud not found in PATH"
print(f"[gcloud container clusters list] {error}", file=_sys.stderr)
except Exception as exc:
error = str(exc)
print(f"[gcloud container clusters list] {error}", file=_sys.stderr)
def _apply():
_lb = getattr(self, "_gke_cluster_listbox", None)
_sv = getattr(self, "_gke_cluster_status_var", None)
if _lb is None:
return
_lb.delete(0, "end")
# Store raw cluster data for later use (get-credentials)
self._gke_clusters_data = clusters
if clusters:
_cur = self.prod_form_vars.get("cloud.clusterName")
_cur = _cur.get().strip() if _cur is not None else ""
for c in clusters:
status_icon = "🟢" if c["status"] == "RUNNING" else "🟡"
check = "✓ " if c["name"] == _cur else " "
label = f"{check}{status_icon} {c['name']} {c['location']}"
_lb.insert("end", label)
if _sv:
_sv.set(f"{len(clusters)} cluster(s) found")
# Auto-select current cluster in listbox
for idx, c in enumerate(clusters):
if c["name"] == _cur:
_lb.selection_set(idx)
_lb.see(idx)
break
else:
if auth_error:
self._gcloud_auth_ok = False
if _sv:
_sv.set("⚠ authentication required")
self._show_gcloud_auth_dialog()
else:
if _sv:
_sv.set(f"✗ {error}" if error else "no clusters found")
self.safe_after(_apply)
threading.Thread(target=_worker, daemon=True).start()
def _activate_selected_gke_cluster(self):
"""Run gcloud container clusters get-credentials for the selected cluster."""
import sys as _sys
import os as _os
listbox = getattr(self, "_gke_cluster_listbox", None)
clusters = getattr(self, "_gke_clusters_data", [])
kube_status_var = getattr(self, "_gke_kubeconfig_status_var", None)
if not listbox or not clusters:
return
sel = listbox.curselection()
if not sel:
return
idx = sel[0]
if idx >= len(clusters):
return
cluster = clusters[idx]
name = cluster["name"]
location = cluster["location"]
_project_id = self.prod_form_vars.get("cloud.projectId")
project_id = _project_id.get().strip() if _project_id is not None else ""
if kube_status_var:
kube_status_var.set(f"⏳ fetching credentials for {name}…")
# Update cloud.clusterName immediately
_cn = self.prod_form_vars.get("cloud.clusterName")
if _cn is not None:
_cn.set(name)
# Refresh listbox checkmarks
_lb = listbox
_lb.delete(0, "end")
for c in clusters:
status_icon = "🟢" if c["status"] == "RUNNING" else "🟡"
check = "✓ " if c["name"] == name else " "
label = f"{check}{status_icon} {c['name']} {c['location']}"
_lb.insert("end", label)
def _worker():
ok = False
message = ""
context_name = ""
cmd = [
"gcloud", "container", "clusters", "get-credentials", name,
"--region", location,
]
if project_id:
cmd += ["--project", project_id]
try:
result = subprocess.run(
cmd,
capture_output=True, text=True, timeout=30,
)
combined = (result.stdout + result.stderr).strip()
if result.returncode == 0:
ok = True
# gcloud prints: kubeconfig entry generated for <name>
# The context name is gke_<project>_<location>_<name>
context_name = f"gke_{project_id}_{location}_{name}"
kube_path = _os.environ.get("KUBECONFIG", _os.path.expanduser("~/.kube/config"))
message = f"✓ KUBECONFIG ready — context: {context_name}"
print(f"[gke get-credentials] {combined}", file=_sys.stderr)
else:
print(f"[gke get-credentials] {combined}", file=_sys.stderr)
first_line = (combined.splitlines() or [""])[0]
message = f"✗ {first_line or 'get-credentials failed'}"
except FileNotFoundError:
message = "✗ gcloud not found in PATH"
print(f"[gke get-credentials] gcloud not found", file=_sys.stderr)
except Exception as exc:
message = f"✗ {exc}"
print(f"[gke get-credentials] {exc}", file=_sys.stderr)
def _apply():
_ksv = getattr(self, "_gke_kubeconfig_status_var", None)
if _ksv:
_ksv.set(message)
if ok and context_name:
# Update KUBECONTEXT in knoe.cfg global section and persist to disk
self.knoe_cfg_data.setdefault("Global", {})["KUBECONTEXT"] = context_name
# Point KUBECONFIG at the file gcloud get-credentials wrote
import os as _os2, pathlib as _pl
default_kube = str(_pl.Path.home() / ".kube" / "config")
if _pl.Path(default_kube).exists():
self.knoe_cfg_data["Global"]["KUBECONFIG"] = default_kube
self._managed_kubeconfig = default_kube
# Save so the context survives screen navigation
try:
self._save_knoe_cfg()
except Exception:
pass
# Always update region to match the selected cluster's location
_rv = self.prod_form_vars.get("cloud.region")
if _rv is not None:
_rv.set(location)
# Rebuild the region combobox values list with ✓ on new region
_combo = getattr(self, "_gcp_region_combo", None)
if _combo is not None:
_existing = list(_combo["values"])
_clean = [v.lstrip("✓ ").strip() for v in _existing]
if location in _clean:
_updated = [
(f"✓ {r}" if r == location else r)
for r in _clean
]
_combo["values"] = _updated
_combo.set(f"✓ {location}")
# Re-evaluate Artifact Registry traffic light now that the
# region is available from the selected cluster's context.
try:
self._check_artifact_registry_async()
except Exception:
pass
self.safe_after(_apply)
threading.Thread(target=_worker, daemon=True).start()
def _fetch_storage_classes(self):
"""Fetch available StorageClasses from the cluster and populate the GCP Storage tab combos."""
import sys as _sys
status_var = getattr(self, "_storage_status_var", None)
combos = getattr(self, "_storage_combos", [])
if not combos:
return
if status_var:
status_var.set("⏳ fetching StorageClasses…")
def _worker():
classes = []
error = ""
try:
result = subprocess.run(
["kubectl", "get", "storageclass", "-o=jsonpath={range .items[*]}{.metadata.name}{'\\n'}{end}"],
capture_output=True, text=True, timeout=20,
)
if result.stderr:
print(f"[kubectl get storageclass] {result.stderr.strip()}", file=_sys.stderr)
if result.returncode == 0:
classes = [l.strip() for l in result.stdout.splitlines() if l.strip()]
else:
error = (result.stderr.splitlines() or ["kubectl error"])[0]
except FileNotFoundError:
error = "kubectl not found in PATH"
print(f"[kubectl get storageclass] {error}", file=_sys.stderr)
except Exception as exc:
error = str(exc)
print(f"[kubectl get storageclass] {exc}", file=_sys.stderr)
def _apply():
_sv = getattr(self, "_storage_status_var", None)
_combos = getattr(self, "_storage_combos", [])
if error:
if _sv:
_sv.set(f"✗ {error}")
return
if _sv:
_sv.set(f"✓ {len(classes)} StorageClass(es) available")
for _cb in _combos:
cur = _cb.get().strip()
_cb["values"] = classes
# Keep current selection if it still exists, else leave as-is
if cur in classes:
_cb.set(cur)
self.safe_after(_apply)
threading.Thread(target=_worker, daemon=True).start()
def _fetch_gcp_regions(self):
"""Fetch available GCP regions in a background thread and populate the region Combobox."""
import sys as _sys
combo = getattr(self, "_gcp_region_combo", None)
status_var = getattr(self, "_gcp_region_status_var", None)
if combo is None:
return
# Skip if auth is already known to be invalid — the auth dialog will retry.
if getattr(self, "_gcloud_auth_ok", None) is False:
if status_var:
status_var.set("⚠ gcloud not authenticated — use Refresh Auth")
return
if status_var:
status_var.set("⏳ loading regions…")
# Read project_id on the main thread before entering the background worker.
_project_id = self.prod_form_vars.get("cloud.projectId")
_project_id = _project_id.get().strip() if _project_id is not None else ""
def _worker():
regions: list[str] = []
error = ""
auth_error = False
# Read project_id from the form — must be done on the main thread before
# entering the worker, but we capture it here via closure.
project_id = _project_id
cmd = ["gcloud", "compute", "regions", "list", "--format=value(name)"]
if project_id:
cmd += ["--project", project_id]
try:
result = subprocess.run(
cmd,
capture_output=True, text=True, timeout=20,
)
if result.returncode == 0:
regions = [r.strip() for r in result.stdout.splitlines() if r.strip()]
regions.sort()
else:
stderr_text = result.stderr.strip()
# Always route raw gcloud errors to stderr — never paint them in the UI.
if stderr_text:
print(f"[gcloud compute regions list] {stderr_text}", file=_sys.stderr)
first_line = (stderr_text.splitlines() or [""])[0]
auth_keywords = ("reauthentication", "auth token", "not authenticated",
"login", "credential")
if any(kw in first_line.lower() for kw in auth_keywords):
auth_error = True
error = "Authentication required"
else:
# Route full error to stderr; show a concise label in the UI.
if not project_id and "required property [project]" in stderr_text.lower():
error = "project ID not set — enter Project ID above and click Reload"
else:
error = first_line or "gcloud error"
except FileNotFoundError:
error = "gcloud not found in PATH"
print(f"[gcloud compute regions list] {error}", file=_sys.stderr)
except Exception as exc:
error = str(exc)
print(f"[gcloud compute regions list] {error}", file=_sys.stderr)
def _apply():
_combo = getattr(self, "_gcp_region_combo", None)
_sv = getattr(self, "_gcp_region_status_var", None)
if _combo is None:
return
if regions:
_cur = self.prod_form_vars["cloud.region"].get().strip()
display = [
f"✓ {r}" if r == _cur else r for r in regions
]
_combo["values"] = display
if _sv:
_sv.set(f"{len(regions)} regions available")
else:
if auth_error:
self._gcloud_auth_ok = False
if _sv:
_sv.set("⚠ authentication required")
self._show_gcloud_auth_dialog()
else:
if _sv:
_sv.set(f"✗ {error}" if error else "no regions returned")
def _on_region_selected(event):
raw = _combo.get()
clean = raw.lstrip("✓ ").strip()
self.prod_form_vars["cloud.region"].set(clean)
_regs = [v.lstrip("✓ ").strip() for v in (_combo["values"] or [])]
_combo["values"] = [
f"✓ {r}" if r == clean else r for r in _regs
]
_combo.bind("<<ComboboxSelected>>", _on_region_selected)
self.safe_after(_apply)
threading.Thread(target=_worker, daemon=True).start()
def _refresh_gcloud_auth_status(self):
"""Run gcloud auth print-access-token in a background thread and update the status label.
On failure, routes the gcloud error to stderr and opens a user-friendly re-auth dialog.
"""
import sys as _sys
def _worker():
auth_ok = False
status = ""
stderr_text = ""
try:
result = subprocess.run(
["gcloud", "auth", "print-access-token"],
capture_output=True, text=True, timeout=10,
)
stderr_text = result.stderr.strip()
if result.returncode == 0 and result.stdout.strip():
acct_result = subprocess.run(
["gcloud", "config", "get-value", "account"],
capture_output=True, text=True, timeout=5,
)
account = acct_result.stdout.strip() or "authenticated"
status = f"✓ {account}"
auth_ok = True
else:
# Route raw gcloud error to stderr — never paint it in the UI.
if stderr_text:
print(f"[gcloud auth print-access-token] {stderr_text}", file=_sys.stderr)
status = "✗ not authenticated — click Refresh Auth or run: gcloud auth login"
except FileNotFoundError:
stderr_text = "gcloud not found in PATH"
print(f"[gcloud auth print-access-token] {stderr_text}", file=_sys.stderr)
status = "✗ gcloud not found in PATH"
except Exception as exc:
stderr_text = str(exc)
print(f"[gcloud auth print-access-token] {stderr_text}", file=_sys.stderr)
status = f"✗ error: {exc}"
def _apply():
self._gcloud_auth_ok = auth_ok
var = getattr(self, "_gcloud_auth_status_var", None)
if var is not None:
var.set(status)
if not auth_ok:
self._show_gcloud_auth_dialog()
try:
self.safe_after(_apply)
except Exception:
pass
threading.Thread(target=_worker, daemon=True).start()
def _show_gcloud_auth_dialog(self):
"""Show a user-friendly modal dialog when gcloud auth is invalid.
Offers a Login button (opens browser via gcloud auth login) and a Retry button.
Streams gcloud output into an in-dialog text area so the auth URL is always visible.
"""
import tkinter as _tk
from tkinter import scrolledtext as _st
import subprocess as _sp
import sys as _sys
import threading as _thr
# Only show one dialog at a time.
existing = getattr(self, "_gcloud_auth_dialog", None)
if existing is not None:
try:
if existing.winfo_exists():
existing.lift()
return
else:
self._gcloud_auth_dialog = None
except Exception:
self._gcloud_auth_dialog = None
# Find a valid parent window via self.root (set by the base screen).
parent = getattr(self, "root", None)
if parent is None or not parent.winfo_exists():
return
dlg = _tk.Toplevel(parent)
dlg.title("GCP Authentication Required")
dlg.resizable(True, True)
dlg.geometry("580x420")
dlg.minsize(520, 360)
dlg.grab_set()
self._gcloud_auth_dialog = dlg
def _on_close():
self._gcloud_auth_dialog = None
dlg.destroy()
dlg.protocol("WM_DELETE_WINDOW", _on_close)
pad = dict(padx=18, pady=6)
_tk.Label(
dlg,
text="🔑 gcloud Authentication Required",
font=("SF Pro Text", 13, "bold"),
fg="#b07d00",
).pack(padx=18, pady=(18, 4))
_tk.Label(
dlg,
text=(
"Your gcloud credentials have expired or are not active.\n"
"Click Login with gcloud to authenticate.\n"
"A browser window will open — if it does not, copy the URL shown below."
),
font=("SF Pro Text", 11),
justify="left",
).pack(**pad)
# Scrolled output area — shows the auth URL so user can copy/paste if needed.
out_frame = _tk.Frame(dlg)
out_frame.pack(fill="x", padx=18, pady=(4, 0))
out_box = _st.ScrolledText(
out_frame,
height=7,
wrap="word",
font=("Menlo", 10),
state="disabled",
bg="#f5f5f0",
fg="#1a1a1a",
insertbackground="#1a1a1a",
relief="solid",
bd=1,
)
out_box.pack(fill="x")
status_var = _tk.StringVar(value="")
status_lbl = _tk.Label(dlg, textvariable=status_var,
font=("SF Pro Text", 9), fg="#888888")
status_lbl.pack(padx=18, pady=(4, 0))
def _append(text):
"""Append text to the output box from any thread."""
def _ui():
try:
out_box.config(state="normal")
out_box.insert("end", text)
out_box.see("end")
out_box.config(state="disabled")
except Exception:
pass
try:
self.safe_after(_ui)
except Exception:
pass
def _do_login():
login_btn.config(state="disabled", text="Authenticating…")
status_var.set("Opening browser — follow the prompts, then click Retry.")
# Clear previous output.
out_box.config(state="normal")
out_box.delete("1.0", "end")
out_box.config(state="disabled")
def _run():
try:
proc = _sp.Popen(
["gcloud", "auth", "login"],
stdout=_sp.PIPE,
stderr=_sp.STDOUT,
text=True,
)
for line in proc.stdout:
print(line, end="", file=_sys.stderr)
_append(line)
proc.wait()
rc = proc.returncode
msg = "✓ Login completed — click Retry to continue." if rc == 0 \
else f"⚠ gcloud exited with code {rc}. Check output above."
def _done():
status_var.set(msg)
login_btn.config(state="normal", text="Login with gcloud")
try:
self.safe_after(_done)
except Exception:
pass
except FileNotFoundError:
msg = "✗ gcloud not found in PATH."
print(f"[gcloud auth login] {msg}", file=_sys.stderr)
_append(msg + "\n")
try:
self.safe_after(lambda: login_btn.config(state="normal", text="Login with gcloud"))
except Exception:
pass
except Exception as exc:
print(f"[gcloud auth login] {exc}", file=_sys.stderr)
_append(f"{exc}\n")
try:
self.safe_after(lambda: login_btn.config(state="normal", text="Login with gcloud"))
except Exception:
pass
_thr.Thread(target=_run, daemon=True).start()
def _do_retry():
_on_close()
self._gcloud_auth_ok = None
self._refresh_gcloud_auth_status()
self.safe_after(self._fetch_gcp_regions, delay=1200)
btn_frame = _tk.Frame(dlg)
btn_frame.pack(padx=18, pady=(10, 18))
login_btn = _tk.Button(
btn_frame, text="Login with gcloud",
command=_do_login,
font=("SF Pro Text", 11, "bold"),
bg="#1558b0", fg="#ffffff", activebackground="#0d3d80",
activeforeground="#ffffff", relief="raised", bd=2,
padx=14, pady=7,
cursor="hand2",
)
login_btn.pack(side="left", padx=(0, 10))
dlg.update_idletasks()
_tk.Button(
btn_frame, text="Retry",
command=_do_retry,
font=("SF Pro Text", 11),
relief="flat",
padx=12, pady=6,
).pack(side="left", padx=(0, 10))
_tk.Button(
btn_frame, text="Dismiss",
command=_on_close,
font=("SF Pro Text", 11),
relief="flat",
padx=12, pady=6,
).pack(side="left")
def _prod_apply_payload_to_vars(self, payload: dict | None):
cfg = build_config(payload or {})
values = {
"metadata.name": cfg.metadata.name,
"metadata.environment": cfg.metadata.environment,
"cloud.provider": cfg.cloud.provider,
"cloud.projectId": cfg.cloud.projectId,
"cloud.region": cfg.cloud.region,
"cloud.clusterName": cfg.cloud.clusterName,
"cloud.vpcMode": cfg.cloud.vpcMode,
"cloud.vpcName": cfg.cloud.vpcName or "",
"cloud.subnetName": cfg.cloud.subnetName or "",
"cloud.artifactRegistry": cfg.cloud.artifactRegistry,
"cloud.dnsZone": cfg.cloud.dnsZone,
"kubernetes.namespace": cfg.kubernetes.namespace,
"database.clusterName": cfg.database.clusterName,
"database.postgresVersion": cfg.database.postgresVersion,
"database.instances": str(cfg.database.instances),
"database.storageClass": cfg.database.storageClass,
"database.storageSizeGi": str(cfg.database.storageSizeGi),
"database.appDatabase": cfg.database.appDatabase,
"database.metaDatabase": cfg.database.metaDatabase,
"database.appUser": cfg.database.appUser,
"database.adminUser": cfg.database.adminUser,
"backups.backupBucket": cfg.backups.backupBucket,
"backups.walBucket": cfg.backups.walBucket,
"backups.retentionDays": str(cfg.backups.retentionDays),
"auth.provider": cfg.auth.provider,
"auth.issuer": cfg.auth.issuer,
"auth.clientId": cfg.auth.clientId,
"auth.clientSecret": cfg.auth.clientSecret,
"auth.bootstrapAdminEmail": cfg.auth.bootstrapAdminEmail,
"routing.frontdoorHost": cfg.routing.frontdoorHost,
"routing.platformDomain": cfg.routing.platformDomain,
"routing.tlsMode": cfg.routing.tlsMode,
"migration.sourceEnvironment": cfg.migration.sourceEnvironment,
"migration.mode": cfg.migration.mode,
"migration.sourceHost": cfg.migration.sourceHost,
"migration.sourcePort": str(cfg.migration.sourcePort),
"migration.sourceDatabase": cfg.migration.sourceDatabase,
"migration.sourceUser": cfg.migration.sourceUser,
"migration.sourcePasswordRef": cfg.migration.sourcePasswordRef,
}
for key, value in values.items():
if key not in self.prod_form_vars:
continue
current = self.prod_form_vars[key].get()
# Don't clobber a real value the user entered with a default placeholder.
# Only apply the payload value if the field is currently empty.
if current.strip():
continue
self.prod_form_vars[key].set(value)
self.prod_bool_vars["migration.continuousUntilCutover"].set(
bool(cfg.migration.continuousUntilCutover)
)
def _prod_payload_from_vars(self) -> dict:
self._ensure_prod_config_state()
def _to_int(key: str) -> int:
try:
return int((self.prod_form_vars[key].get() or "").strip())
except Exception:
return 0
def _to_int_text(value, default: int) -> int:
try:
text = (value or "").strip()
if not text:
return int(default)
return int(text)
except Exception:
return int(default)
region_form = self.prod_form_vars["cloud.region"].get().strip()
selected_region = region_form
db_storage_class = self.prod_form_vars["database.storageClass"].get().strip() or "premium-rwo"
pgdata_storage_class = db_storage_class
wal_storage_class = pgdata_storage_class
if pgdata_storage_class not in {"premium-rwo", "standard-rwo"}:
pgdata_storage_class = "premium-rwo"
if wal_storage_class not in {"premium-rwo", "standard-rwo"}:
wal_storage_class = pgdata_storage_class
default_pgdata_size = max(1, _to_int("database.storageSizeGi"))
pgdata_size_gi = _to_int_text(str(default_pgdata_size), default_pgdata_size)
default_wal = max(10, int(pgdata_size_gi * 0.25))
wal_size_gi = _to_int_text(str(default_wal), default_wal)
doc = {
"kind": "KnoeProductionConfig",
"metadata": {
"ecosystemId": 0,
"name": self.prod_form_vars["metadata.name"].get().strip(),
"environment": self.prod_form_vars["metadata.environment"].get().strip(),
},
"spec": {
"cloud": {
"provider": self.prod_form_vars["cloud.provider"].get().strip(),
"projectId": self.prod_form_vars["cloud.projectId"].get().strip(),
"region": selected_region,
"clusterName": self.prod_form_vars["cloud.clusterName"].get().strip(),
"vpcMode": self.prod_form_vars["cloud.vpcMode"].get().strip(),
"vpcName": self.prod_form_vars["cloud.vpcName"].get().strip() or None,
"subnetName": self.prod_form_vars["cloud.subnetName"].get().strip() or None,
"artifactRegistry": self.prod_form_vars[
"cloud.artifactRegistry"
].get().strip(),
"dnsZone": self.prod_form_vars["cloud.dnsZone"].get().strip(),
},
"kubernetes": {
"namespace": self.prod_form_vars["kubernetes.namespace"].get().strip()
},
"database": {
"clusterName": self.prod_form_vars["database.clusterName"].get().strip(),
"postgresVersion": self.prod_form_vars[
"database.postgresVersion"
].get().strip(),
"instances": _to_int("database.instances"),
"storageClass": pgdata_storage_class,
"storageSizeGi": pgdata_size_gi,
"pgdataStorageClass": pgdata_storage_class,
"walStorageClass": wal_storage_class,
"pgdataSizeGi": pgdata_size_gi,
"walSizeGi": wal_size_gi,
"appDatabase": self.prod_form_vars["database.appDatabase"].get().strip(),
"metaDatabase": self.prod_form_vars["database.metaDatabase"].get().strip(),
"appUser": self.prod_form_vars["database.appUser"].get().strip(),
"adminUser": self.prod_form_vars["database.adminUser"].get().strip(),
},
"backups": {
"backupBucket": self.prod_form_vars["backups.backupBucket"].get().strip(),
"walBucket": self.prod_form_vars["backups.walBucket"].get().strip(),
"retentionDays": _to_int("backups.retentionDays"),
},
"auth": {
"provider": self.prod_form_vars["auth.provider"].get().strip(),
"issuer": self.prod_form_vars["auth.issuer"].get().strip(),
"clientId": self.prod_form_vars["auth.clientId"].get().strip(),
"clientSecret": self.prod_form_vars["auth.clientSecret"].get().strip(),
"bootstrapAdminEmail": self.prod_form_vars[
"auth.bootstrapAdminEmail"
].get().strip(),
},
"routing": {
"frontdoorHost": self.prod_form_vars[
"routing.frontdoorHost"
].get().strip(),
"platformDomain": self.prod_form_vars[
"routing.platformDomain"
].get().strip(),
"tlsMode": self.prod_form_vars["routing.tlsMode"].get().strip(),
},
"migration": {
"sourceEnvironment": self.prod_form_vars[
"migration.sourceEnvironment"
].get().strip(),
"mode": self.prod_form_vars["migration.mode"].get().strip(),
"sourceHost": self.prod_form_vars["migration.sourceHost"].get().strip(),
"sourcePort": _to_int("migration.sourcePort"),
"sourceDatabase": self.prod_form_vars[
"migration.sourceDatabase"
].get().strip(),
"sourceUser": self.prod_form_vars["migration.sourceUser"].get().strip(),
"sourcePasswordRef": self.prod_form_vars[
"migration.sourcePasswordRef"
].get().strip(),
"continuousUntilCutover": self.prod_bool_vars[
"migration.continuousUntilCutover"
].get(),
},
},
}
return to_api_payload(build_config(doc))
def _prod_format_messages(self, errors: list[str], warnings: list[str]) -> str:
lines = []
if errors:
lines.append("Errors:")
lines.extend([f"- {line}" for line in errors])
if warnings:
if lines:
lines.append("")
lines.append("Warnings:")
lines.extend([f"- {line}" for line in warnings])
return "\n".join(lines)
def _prod_push_preview_widgets(self):
combined = [
"=== Canonical YAML Config Preview ===",
self.prod_preview_yaml or "(no plan yet)",
"=== OpenTofu Variables Preview ===",
self.prod_preview_tf_vars or "(no plan yet)",
"=== install.py Module Plan ===",
self.prod_preview_install_plan or "(no plan yet)",
"=== Warnings / Errors ===",
self.prod_tab_messages.get() or "(none)",
]
preview_text = "\n".join(combined).strip() + "\n"
widget = getattr(self, "_prod_preview_text_widget", None)
if widget is not None:
try:
widget.configure(state="normal")
widget.delete("1.0", tk.END)
widget.insert("1.0", preview_text)
widget.configure(state="disabled")
except Exception:
pass
log_widget = getattr(self, "_prod_logs_text_widget", None)
if log_widget is not None:
try:
log_widget.configure(state="normal")
log_widget.delete("1.0", tk.END)
log_widget.insert("1.0", (self.prod_apply_logs or "") + "\n")
log_widget.configure(state="disabled")
except Exception:
pass
def _prod_sync_status_and_logs(self):
self._ensure_prod_config_state()
try:
status = self.prod_config_api.get_status() or {}
except Exception:
status = {}
try:
logs = (self.prod_config_api.get_logs() or {}).get("logs", [])
except Exception:
logs = []
phase = (status.get("phase") or "unknown").strip()
state = (status.get("state") or "unknown").strip()
message = (status.get("message") or "").strip()
self.prod_apply_status.set(f"{phase}/{state}: {message}".strip())
self.prod_apply_logs = "\n".join(logs[-300:])
self._prod_push_preview_widgets()
def _prod_put_config(self, show_dialog: bool = False) -> dict:
self._ensure_prod_config_state()
result = self.prod_config_api.put_prod_config(self._prod_payload_from_vars())
# Persist GKE node pool config to [Initialize Cluster] section of knoe.cfg
_gke_sec = self.knoe_cfg_data.setdefault("Initialize Cluster", {})
_disk_type = self.prod_form_vars["init_cluster.db_boot_disk_type"].get().strip()
_disk_size = self.prod_form_vars["init_cluster.db_boot_disk_size_gb"].get().strip()
if _disk_type:
_gke_sec["DB_BOOT_DISK_TYPE"] = _disk_type
if _disk_size:
_gke_sec["DB_BOOT_DISK_SIZE_GB"] = _disk_size
cfg_doc = result.get("config") or {}
if cfg_doc:
self._prod_apply_payload_to_vars(cfg_doc)
errors = list(result.get("errors") or [])
warnings = list(result.get("warnings") or [])
self.prod_tab_messages.set(self._prod_format_messages(errors, warnings))
self._prod_push_preview_widgets()
if show_dialog:
if errors:
messagebox.showwarning("Production Config", "\n".join(errors))
else:
note = "Production config saved."
if warnings:
note += "\n\nWarnings:\n" + "\n".join(warnings)
messagebox.showinfo("Production Config", note)
return result
def _on_prod_plan(self):
self._ensure_prod_config_state()
put_result = self._prod_put_config(show_dialog=False)
if put_result.get("errors"):
self._prod_sync_status_and_logs()
return
result = self.prod_config_api.post_plan(self._prod_payload_from_vars())
self.prod_preview_yaml = result.get("yaml", "")
self.prod_preview_tf_vars = json.dumps(
result.get("opentofuVars") or {}, indent=2, sort_keys=True
)
self.prod_preview_install_plan = "\n".join(
f"- {line}" for line in (result.get("installPlan") or [])
)
self.prod_tab_messages.set(
self._prod_format_messages(
list(result.get("errors") or []), list(result.get("warnings") or [])
)
)
self._prod_sync_status_and_logs()
def _on_prod_apply(self):
self._ensure_prod_config_state()
put_result = self._prod_put_config(show_dialog=False)
if put_result.get("errors"):
self._prod_sync_status_and_logs()
return
result = self.prod_config_api.post_apply(self._prod_payload_from_vars())
self.prod_preview_yaml = result.get("yaml", "")
self.prod_preview_tf_vars = json.dumps(
result.get("opentofuVars") or {}, indent=2, sort_keys=True
)
self.prod_preview_install_plan = "\n".join(
f"- {line}" for line in (result.get("installPlan") or [])
)
self.prod_tab_messages.set(
self._prod_format_messages(
list(result.get("errors") or []), list(result.get("warnings") or [])
)
)
self._prod_sync_status_and_logs()
def _on_prod_refresh_status(self):
self._ensure_prod_config_state()
self._prod_sync_status_and_logs()
def _validate_and_save_cluster_config(self) -> bool:
"""Verify the config then write the values to knoe.cfg."""
env_key = self._cluster_env_key()
prev_service_ns = ""
try:
prev_service_ns = (
(self.knoe_cfg_data.get("Global", {}) or {})
.get("SERVICE_NAMESPACE", "")
.strip()
)
except Exception:
prev_service_ns = ""
if env_key == "dev":
cluster_val = self.selected_k3d_cluster.get() or "knoe-dev-cluster"
if not cluster_val:
messagebox.showwarning(
"Validation", "Please select or create a k3d cluster."
)
return False
# Context switch (if context matches cluster name)
try:
subprocess.run(
["kubectl", "config", "use-context", f"k3d-{cluster_val}"],
capture_output=True,
text=True,
)
except Exception:
pass
elif env_key == "service":
# Service deployments can authenticate via an existing kubeconfig/context
# (preferred), or via a token-based generated kubeconfig.
kubeconfig = _find_kubeconfig_file()
has_token_flow = bool((self.k3s_server_url.get() or "").strip()) and bool(
(self.k3s_token.get() or "").strip()
)
if not kubeconfig and not has_token_flow:
messagebox.showwarning(
"Validation",
"Missing cluster credentials. Please select a working Kubernetes context (kubectx) "
"or provide K3s server URL + token.",
)
return False
cluster_val = self.cluster_env.get()
elif env_key == "prod":
path = self.prod_artifacts_path.get().strip()
if not path:
messagebox.showwarning(
"Validation", "Please specify an artifact staging directory."
)
return False
self._ensure_prod_config_state()
put_result = self._prod_put_config(show_dialog=False)
prod_errors = list(put_result.get("errors") or [])
if prod_errors:
messagebox.showwarning(
"Validation",
"Production config validation failed:\n\n"
+ "\n".join(prod_errors),
)
return False
cfg_vars = _collect_cfg_vars_from_data(getattr(self, "knoe_cfg_data", None))
expanded_path = _expand_path_expr(path, cfg_vars)
p = Path(expanded_path).expanduser()
if not p.exists():
try:
p.mkdir(parents=True, exist_ok=True)
except Exception as e:
messagebox.showerror(
"Error", f"Failed to create directory {expanded_path or path}: {e}"
)
return False
cluster_val = self.cluster_env.get()
else:
cluster_val = self.cluster_env.get()
# Capture cluster info for config.
# ENVIRONMENT here is used as an env/mode hint; keep it as the selected env key.
self.knoe_cfg_data["Initialize Cluster"]["ENVIRONMENT"] = (
self.cluster_env.get() or ""
).strip()
self.knoe_cfg_data["Initialize Cluster"]["K3S_SERVER_URL"] = (
self.k3s_server_url.get() or ""
).strip()
self.knoe_cfg_data["Initialize Cluster"]["K3S_TOKEN"] = _encrypt_cfg_secret(
self.k3s_token.get() or ""
)
self.knoe_cfg_data["Global"][
"SERVICE_NAMESPACE"
] = self._get_service_namespace()
new_service_ns = self.knoe_cfg_data["Global"]["SERVICE_NAMESPACE"]
self.knoe_cfg_data["Optional Features"]["SUPABASE_ENABLED"] = str(
self.supabase_enabled.get()
)
self.knoe_cfg_data["Optional Features"]["GITOPS_ENABLED"] = str(
self.gitops_enabled.get()
)
self.knoe_cfg_data["Optional Features"]["KERBEROS_ENABLED"] = str(
self.kerberos_enabled.get()
)
self.knoe_cfg_data["Optional Features"]["AT_REST_ENCRYPTION_ENABLED"] = str(
self.at_rest_encryption_enabled.get()
)
if env_key == "prod":
self.knoe_cfg_data["Prod Cluster (k8s)"][
"ARTIFACTS_DIR"
] = self.prod_artifacts_path.get()
try:
prod_payload = self._prod_payload_from_vars()
prod_plan = self.prod_config_api.post_plan(prod_payload)
self.knoe_cfg_data["Prod Cluster (k8s)"][
"PRODUCTION_CONFIG_YAML"
] = prod_plan.get("yaml", "")
self.knoe_cfg_data["Prod Cluster (k8s)"][
"OPENTOFU_VARS"
] = json.dumps(
prod_plan.get("opentofuVars") or {}, sort_keys=True, separators=(",", ":")
)
self.knoe_cfg_data["Prod Cluster (k8s)"][
"INSTALL_PLAN"
] = "\n".join(prod_plan.get("installPlan") or [])
self.knoe_cfg_data["Prod Cluster (k8s)"][
"PLAN_MESSAGES"
] = self.prod_tab_messages.get()
except Exception:
pass
# Save config
self._save_knoe_cfg()
# Namespace change cleanup (prevents collisions when switching namespaces).
try:
prev = (prev_service_ns or "").strip()
new = (new_service_ns or "").strip()
if prev and new and prev != new:
do_cleanup = False
try:
do_cleanup = messagebox.askyesno(
"Namespace Changed",
(
f"Common Core Services namespace changed from '{prev}' to '{new}'.\n\n"
f"Reset the OLD namespace '{prev}' to avoid resource collisions?\n\n"
"This deletes workloads/services in that namespace (PVCs are preserved)."
),
)
except Exception:
do_cleanup = False
if do_cleanup:
def _cleanup_worker():
try:
self._set_cluster_env_message(
f"Resetting old namespace: {prev}",
"#ff9500",
clear_after_ms=3000,
)
try:
self._ensure_k3s_kubeconfig_merged()
except Exception:
pass
server, token = ("", "")
try:
server, token = self._k3s_connection_info()
except Exception:
server, token = ("", "")
controller = getattr(self, "controller", None)
project_root = getattr(controller, "project_root", None)
if project_root:
_reset_k3s_namespace(project_root, prev, server, token)
self._set_cluster_env_message(
f"Old namespace reset complete: {prev}",
"#34c759",
clear_after_ms=4000,
)
except Exception as e:
self._set_cluster_env_message(
f"Namespace reset failed: {e}",
"#ff3b30",
clear_after_ms=6000,
)
threading.Thread(target=_cleanup_worker, daemon=True).start()
except Exception:
pass
# On save, immediately run a services status refresh so the UI reflects the
# newly persisted namespace/cluster settings.
try:
self._verify_k3s_services()
except Exception:
pass
return True
def _cluster_status_snapshot(self) -> dict:
env = self._cluster_env_key()
cluster_ok = False
if env == "dev":
cluster_name = self.selected_k3d_cluster.get() or "knoe-dev-cluster"
try:
res = subprocess.run(
["k3d", "cluster", "list", "--no-headers"],
capture_output=True,
text=True,
)
cluster_ok = cluster_name in (res.stdout or "")
except Exception:
cluster_ok = False
cluster_msg = (
f"K3D Cluster ({cluster_name}): Running"
if cluster_ok
else f"K3D Cluster ({cluster_name}): Not found"
)
else:
cluster_ok, cluster_msg = self._check_k8s_cluster(env)
cluster_fill = "#34c759" if cluster_ok else "#ff9f0a"
common_ok = cluster_ok
common_msg = (
"Common Services: Ready to deploy"
if common_ok
else "Common Services: Not ready"
)
common_fill = "#34c759" if common_ok else "#ff9f0a"
return {
"env": env,
"cluster_ok": cluster_ok,
"cluster_msg": cluster_msg,
"cluster_fill": cluster_fill,
"common_ok": common_ok,
"common_msg": common_msg,
"common_fill": common_fill,
}
def _cluster_ready_for_navigation(self) -> bool:
status = self._cluster_status_snapshot()
if not status["cluster_ok"]:
try:
messagebox.showerror(
"Cluster",
"Cluster is not reachable. Please verify your cluster and try again.",
)
except Exception:
pass
return False
return True
def check_cluster_status_async(self):
def worker():
notice_active = False
self._set_cluster_env_message("Collect Status", "#34c759")
status = self._cluster_status_snapshot()
self._set_cluster_env_message("Verify State", "#34c759")
kong_status = None
if status.get("cluster_ok"):
kong_status = self._dashboard_kong_status(status.get("env"))
self._set_cluster_env_message("Observe", "#34c759")
# Observe-only: do not mutate the cluster automatically.
anomalies = []
try:
if kong_status and not kong_status.get("ok", True):
anomalies.append("dashboard")
except Exception:
pass
try:
if self._authority_context_missing():
anomalies.append("authority")
except Exception:
pass
if anomalies:
self._set_cluster_env_message(
"Notice: issues detected. Click Repair to reconcile.",
"#ff9f0a",
clear_after_ms=6000,
)
notice_active = True
def update_ui():
if hasattr(self, "k3d_status_label"):
self.bg_canvas.itemconfig(
self.k3d_status_label,
text=status["cluster_msg"],
fill=status["cluster_fill"],
)
if hasattr(self, "common_services_status_label"):
self.bg_canvas.itemconfig(
self.common_services_status_label,
text=status["common_msg"],
fill=status["common_fill"],
)
if hasattr(self, "_init_cluster_button"):
try:
self._init_cluster_button.configure(text="Save")
except Exception:
pass
self.root.after(0, update_ui)
# Refresh Artifact Registry light whenever cluster status is checked (prod only).
try:
self._check_artifact_registry_async()
except Exception:
pass
# Clear message after a short delay if no notice is active
if not notice_active:
self._set_cluster_env_message("", "#34c759", clear_after_ms=2000)
threading.Thread(target=worker, daemon=True).start()
def _services_status_snapshot(self) -> dict:
opentofu_ok = self._check_opentofu_health()
opentofu_msg = "OpenTofu: Running" if opentofu_ok else "OpenTofu: Not reachable"
opentofu_fill = "#34c759" if opentofu_ok else "#ff9f0a"
return {
"opentofu_msg": opentofu_msg,
"opentofu_fill": opentofu_fill,
}
def check_services_status_async(self):
def worker():
status = self._services_status_snapshot()
def update_ui():
if hasattr(self, "_db_opentofu_status_label"):
self.bg_canvas.itemconfig(
self._db_opentofu_status_label,
text=status["opentofu_msg"],
fill=status["opentofu_fill"],
)
self.root.after(0, update_ui)
threading.Thread(target=worker, daemon=True).start()
def _check_artifact_registry_async(self):
"""Check GCP Artifact Registry availability and persist the result.
Updates the ``_artifact_registry_indicator`` oval on the Cluster Environment
screen and stores ``ARTIFACT_REGISTRY_AVAILABLE`` in ``knoe_cfg_data["Global"]``
so the Database Build screen can read it without re-checking.
"""
def worker():
ok = False
msg = "Artifact Registry: unavailable"
registry_url = ""
try:
from knoe.core.env import _deployment_mode_from_env # local import to avoid cycles
env_key = self._cluster_env_key()
mode = _deployment_mode_from_env(env_key)
if mode != "k8s":
# Only relevant for GKE/prod; skip silently for other envs.
return
gcp = self.knoe_cfg_data.get("GCP", {}) or {}
project_id = (gcp.get("project_id") or gcp.get("PROJECT_ID") or "").strip().strip('"')
kubecontext = (
(self.knoe_cfg_data.get("Global", {}) or {}).get("KUBECONTEXT") or ""
).strip()
region = ""
if kubecontext.startswith("gke_"):
parts = kubecontext.split("_", 3)
if len(parts) >= 3:
region = parts[2]
ar_repo = (
(self.knoe_cfg_data.get("Global", {}) or {})
.get("SERVICE_NAMESPACE", "knoe-system")
.strip() or "knoe-system"
)
if project_id and region:
result = subprocess.run(
[
"gcloud", "artifacts", "repositories", "describe", ar_repo,
"--project", project_id,
"--location", region,
"--format", "value(name)",
],
capture_output=True,
text=True,
timeout=15,
)
ok = result.returncode == 0
registry_url = f"{region}-docker.pkg.dev/{project_id}/{ar_repo}"
msg = (
f"Artifact Registry: {registry_url}"
if ok
else f"Artifact Registry: {registry_url} (unavailable)"
)
elif project_id:
msg = "Artifact Registry: region not set"
else:
msg = "Artifact Registry: GCP project not configured"
except Exception as exc:
msg = f"Artifact Registry: error ({exc})"
ok = False
# Persist so the Database Build screen can read without re-checking.
try:
global_cfg = self.knoe_cfg_data.setdefault("Global", {})
global_cfg["ARTIFACT_REGISTRY_AVAILABLE"] = "true" if ok else "false"
if registry_url:
global_cfg["ARTIFACT_REGISTRY"] = registry_url
self._save_knoe_cfg()
except Exception:
pass
fill = "#34c759" if ok else "#ff3b30"
def update_ui():
try:
if hasattr(self, "_artifact_registry_indicator"):
self.bg_canvas.itemconfig(
self._artifact_registry_indicator, fill=fill
)
if hasattr(self, "_artifact_registry_label"):
self.bg_canvas.itemconfig(
self._artifact_registry_label, text=msg
)
except Exception:
pass
self.root.after(0, update_ui)
threading.Thread(target=worker, daemon=True).start()
def ensure_cluster_ready(self):
self._action_flags["init_cluster.start_cluster"] = True
# Implementation of cluster creation/startup
def worker():
cluster_env = self._cluster_env_key()
if cluster_env == "dev":
# 1. Start Docker if not running
if not self.controller.check_docker_running():
# Attempt to start Docker on macOS
subprocess.run(["open", "-a", "Docker"], capture_output=True)
# Wait for it to start
for _ in range(30):
time.sleep(2)
if self.controller.check_docker_running():
break
if not self.controller.check_docker_running():
self.root.after(
0,
lambda: messagebox.showerror(
"Docker",
"Could not start Docker. Please start it manually.",
),
)
return
# 2. Manage or verify cluster
cluster_name = "knoe-dev-cluster"
res = subprocess.run(
["k3d", "cluster", "list", "--no-headers"],
capture_output=True,
text=True,
)
res_stdout = res.stdout or ""
if self._reset_cluster:
subprocess.run(
["k3d", "cluster", "delete", cluster_name], capture_output=True
)
res_stdout = ""
self._reset_cluster = False
if cluster_name not in res_stdout:
# Create it
# Default args based on README.md
knoe_data = str(self._resolve_env_dir("PROLE_DATA", "data"))
volume_args = _k3d_knoe_data_volume_args(knoe_data)
cmd = [
"k3d",
"cluster",
"create",
cluster_name,
"-a",
"2",
] + volume_args
reg_args = []
try:
if self.ensure_local_registry_available():
reg_args = ["--registry-use", "k3d-knoe-registry:5000"]
except Exception:
reg_args = []
cmd += reg_args + ["--api-port", "0.0.0.0:6443"]
# Run in terminal or capture output? Let's use a console window later.
# For now, run it and update status.
subprocess.run(cmd, capture_output=True)
else:
# Start it if it's stopped
subprocess.run(
["k3d", "cluster", "start", cluster_name], capture_output=True
)
self.check_cluster_status_async()
self.root.after(
0,
lambda: messagebox.showinfo(
"Cluster", f"Cluster {cluster_name} is ready."
),
)
else:
if self._reset_cluster and cluster_env in ("service", "k3s"):
ns = (self._get_service_namespace() or "").strip() or "default"
server, token = self._k3s_connection_info()
_reset_k3s_namespace(
self.controller.project_root, ns, server, token
)
self._reset_cluster = False
ok, msg = self._check_k8s_cluster(cluster_env)
if not ok:
self.root.after(0, lambda: messagebox.showerror("Cluster", msg))
return
self.check_cluster_status_async()
self.root.after(0, lambda: messagebox.showinfo("Cluster", msg))
threading.Thread(target=worker, daemon=True).start()
def _verify_k3s_services(self):
if not self._k3s_service_status_console:
self._stop_k3s_service_status_updates()
return
try:
self._k3s_service_status_console.clear()
except Exception:
pass
try:
if self._k3s_service_notebook and self._k3s_service_status_tab:
self._k3s_service_notebook.select(self._k3s_service_status_tab)
except Exception:
pass
self._start_k3s_service_status_updates()
def _start_k3s_service_status_updates(self):
if not self._k3s_status_active:
self._k3s_status_active = True
if self._k3s_status_after_id:
try:
self.root.after_cancel(self._k3s_status_after_id)
except Exception:
pass
self._k3s_status_after_id = None
self._schedule_k3s_service_status_update(0)
def _stop_k3s_service_status_updates(self):
self._k3s_status_active = False
self._k3s_status_inflight = False
if self._k3s_status_after_id:
try:
self.root.after_cancel(self._k3s_status_after_id)
except Exception:
pass
self._k3s_status_after_id = None
def _schedule_k3s_service_status_update(self, delay_ms: int):
if not self._k3s_status_active:
return
if not self.root or not self.root.winfo_exists():
return
try:
self._k3s_status_after_id = self.root.after(
delay_ms, self._run_k3s_service_status_once
)
except Exception:
self._k3s_status_after_id = None
def _run_k3s_service_status_once(self):
if self._k3s_status_inflight:
return
if not self._k3s_service_status_console:
self._k3s_status_inflight = False
self.safe_after(lambda: self._schedule_k3s_service_status_update(30000))
return
self._k3s_status_inflight = True
console = self._k3s_service_status_console
console.clear()
console.write(
f"Service status refresh @ {time.strftime('%Y-%m-%d %H:%M:%S')}\n\n"
)
def worker():
collected_lines = []
try:
ns = self._get_service_namespace()
env = self._script_env_for_namespace(ns, cluster_role="app")
env["KNOE_MODE"] = self._deployment_mode()
if not env.get("KUBECONFIG"):
console.write(
"KUBECONFIG not generated. Check cluster credentials in knoe.cfg.\n"
)
return
verify_args = ["-n", ns, "verify"]
def _on_line(line):
console.write(line)
collected_lines.append(line)
rc = self.controller.run_script(
"init_common_services.sh",
args=verify_args,
env=env,
on_line=_on_line,
)
console.write(f"\nVerification Exit status: {rc}\n")
# Parse per-component traffic light status from [OK]/[FAIL] lines
component_status = self._parse_service_status_lines(collected_lines)
all_green = rc == 0
def _update_traffic_lights():
self._all_services_green = all_green
# Ensure common services success flag is synced for navigation
self._common_services_success = all_green
lights = getattr(self, "_service_traffic_lights", {})
for comp_key, indicator_id in lights.items():
if comp_key in component_status:
color = (
"#34c759" if component_status[comp_key] else "#ff3b30"
)
else:
color = "#34c759" if all_green else "#8e8e93"
try:
self.bg_canvas.itemconfig(
indicator_id, fill=color, outline=color
)
except Exception:
pass
# Update footer to enable Next button if everything is green
self.update_footer()
self.safe_after(_update_traffic_lights)
finally:
self._k3s_status_inflight = False
self.safe_after(lambda: self._schedule_k3s_service_status_update(30000))
threading.Thread(target=worker, daemon=True).start()
def _parse_service_status_lines(self, lines: list[str]) -> dict[str, bool]:
"""Parse [OK]/[FAIL] lines from status_common_services.sh output.
Returns a dict mapping component keys (registry, certmgr,
garage, kong, openbao, opentofu) to True (healthy) or False (unhealthy).
"""
# Map resource names to traffic light component keys
name_map = {
"registry": "registry",
"opentofu": "opentofu",
"garage": "garage",
"openbao": "openbao",
"knoe-svc-kong": "kong",
"cert-manager": "certmgr",
"cert-manager-cainjector": "certmgr",
"cert-manager-webhook": "certmgr",
}
# Start with all components healthy (True); any FAIL sets to False
result: dict[str, bool] = {}
import re as _re
for line in lines:
m = _re.match(r"\[(OK|FAIL)\]\s+(\S+)/(\S+)", line)
if not m:
continue
status_tag = m.group(1)
resource_name = m.group(3)
comp_key = name_map.get(resource_name)
if not comp_key:
continue
if comp_key not in result:
result[comp_key] = True
if status_tag == "FAIL":
result[comp_key] = False
return result
def _deploy_k3s_services(self):
"""Deploy common services (Registry/OpenTofu/Garage) to remote k3s cluster."""
def _deploy():
console = self._k3s_service_deploy_console
if console:
console.clear()
console.write(
f"Deploying common services @ {time.strftime('%Y-%m-%d %H:%M:%S')}\n\n"
)
try:
if self._k3s_service_notebook and self._k3s_service_deploy_tab:
self._k3s_service_notebook.select(self._k3s_service_deploy_tab)
except Exception:
pass
service_ns = self._get_service_namespace()
env = self._script_env_for_namespace(service_ns, cluster_role="app")
env["KNOE_MODE"] = self._deployment_mode()
log_path = self._common_services_log_path()
env["COMMON_SERVICES_INIT_LOG"] = str(log_path)
try:
log_fp = log_path.open("a", encoding="utf-8")
log_fp.write(
f"\n# Common services deploy @ {time.strftime('%Y-%m-%d %H:%M:%S')}\n"
)
log_fp.flush()
except Exception:
log_fp = None
if console:
console.write(f"Log file: {log_path}\n\n")
if not env.get("KUBECONFIG"):
if console:
console.write(
"KUBECONFIG not generated. Check cluster credentials in knoe.cfg.\n"
)
if log_fp:
try:
log_fp.write(
"KUBECONFIG not generated. Check cluster credentials in knoe.cfg.\n"
)
log_fp.close()
except Exception:
pass
self._verify_k3s_services()
return
try:
common_args = ["-n", env["NAMESPACE"]]
if self.kerberos_enabled.get():
common_args.append("-k")
common_args.append("update")
def _log_line(line: str):
if console:
console.write(line)
if log_fp:
try:
log_fp.write(line)
log_fp.flush()
except Exception:
pass
rc = self.controller.run_script(
"init_common_services.sh",
args=common_args,
env=env,
on_line=_log_line,
)
if console:
console.write(f"\nExit status: {rc}\n")
if log_fp:
try:
log_fp.write(f"\nExit status: {rc}\n")
except Exception:
pass
finally:
if log_fp:
try:
log_fp.close()
except Exception:
pass
self._verify_k3s_services()
threading.Thread(target=_deploy, daemon=True).start()
def _ensure_k3s_kubeconfig_merged(self):
"""Ensure a valid kubeconfig is available and merged into ~/.kube/config.
Prefers an existing Ansible-fetched kubeconfig (with client-certificate
auth) over generating a new token-based one.
"""
# 1. Check for existing cert-based kubeconfig first
existing = _find_kubeconfig_file()
if existing:
self._managed_kubeconfig = existing
_merge_kubeconfig(existing)
return
# 2. Fall back to token-based generation
server, token = self._k3s_connection_info()
if not server or not token:
return
kubeconfig_path = str(_write_k3s_kubeconfig(server, token))
self._managed_kubeconfig = kubeconfig_path
_merge_kubeconfig(kubeconfig_path)
def _k3s_connection_info(self) -> tuple[str, str]:
# Check UI fields first; fall back to the shared 3-source resolver.
ui_server = (_safe_str(self.k3s_server_url.get()) or "").strip()
ui_token = (_safe_str(self.k3s_token.get()) or "").strip()
if ui_token:
ui_token = self._resolve_secret_value(ui_token)
ui_token = _normalize_k3s_token(ui_token)
if ui_server and not ui_server.startswith("http"):
ui_server = f"https://{ui_server}"
# If UI fields are populated, use them directly.
if ui_server and ui_token:
return ui_server, ui_token
# Otherwise delegate to the unified resolver (env → cfg → ansible).
resolved_server, resolved_token = _resolve_k3s_connection_fn(
project_root=PROJECT_ROOT,
)
return ui_server or resolved_server, ui_token or resolved_token
def _k3s_connection_raw(self) -> tuple[str, str]:
"""Return (server_url, token) without attempting secret resolution."""
server = (
_safe_str(self.k3s_server_url.get())
or os.environ.get("PROLE_K3S_SERVER")
or os.environ.get("K3S_SERVER_URL")
or ""
).strip()
token = (
_safe_str(self.k3s_token.get())
or os.environ.get("PROLE_K3S_TOKEN")
or os.environ.get("K3S_TOKEN")
or ""
).strip()
if server and not server.startswith("http"):
server = f"https://{server}"
return server, token
Reference in New Issue View Git Blame Copy Permalink