Commit Graph

14 Commits

Author SHA1 Message Date
chrisfu
6741a684dd chore: add GKE auth plugin and kubeconfig handling for improved GKE bootstrap
- Added `gke-gcloud-auth-plugin` as a required dependency for `linux/apt` environments.
- Enhanced GKE context acquisition with writable kubeconfig target resolution.
- Improved failure reporting with detailed messages for GKE dependency and context issues.
- Expanded test coverage for scenarios involving missing auth plugins, kubeconfig errors, and GKE configurations.
2026-04-12 18:51:35 -07:00
chrisfu
797b566663 chore: refactor dependency resolution and auto-install logic
- Replaced static dependency configuration with dynamic resolution using `get_required_dependencies` and `get_required_dependency_ids`.
- Streamlined runtime checks, fallback behaviors, and handling of missing/optional dependencies.
- Hardened GKE context acquisition and validation for Kubernetes clusters.
- Enhanced test coverage for dependency resolution under various deployment modes (`dev`, `prod`, `gke`).
2026-04-12 18:36:39 -07:00
chrisfu
10bfc42bad chore: refactor dependency resolution and auto-install logic
- Replaced static dependency configuration with dynamic resolution using `get_required_dependencies` and `get_required_dependency_ids`.
- Streamlined runtime checks, fallback behaviors, and handling of missing/optional dependencies.
- Hardened GKE context acquisition and validation for Kubernetes clusters.
- Enhanced test coverage for dependency resolution under various deployment modes (`dev`, `prod`, `gke`).
2026-04-12 18:17:35 -07:00
chrisfu
cedcaae2c5 chore: update prole.cfg defaults for dev cluster, refine config overrides
- Updated default paths, namespaces, and cluster names for k3d/dev configuration.
- Added `skip_names` set to exclude `prod.cfg` and `gcp.cfg` from override processing.
- Refined init parameters for Supabase, Kubernetes, and database deployments to align with dev-specific settings.
- Enhanced test coverage for excluded config files (`prod.cfg`, `gcp.cfg`) within `test_prole_conf.py`.
2026-04-12 14:58:55 -07:00
chrisfu
138a01c181 chore: update prole.cfg defaults for dev cluster, refine config overrides
- Updated default paths, namespaces, and cluster names for k3d/dev configuration.
- Added `skip_names` set to exclude `prod.cfg` and `gcp.cfg` from override processing.
- Refined init parameters for Supabase, Kubernetes, and database deployments to align with dev-specific settings.
- Enhanced test coverage for excluded config files (`prod.cfg`, `gcp.cfg`) within `test_prole_conf.py`.
2026-04-12 12:52:43 -07:00
chrisfu
e3c2e625f1 refactor(config): separate k3d k3s and gke config entrypoints
Rename env config files from conf/*/prole.cfg to conf/k3d.cfg, conf/k3s.cfg, and conf/gke.cfg. Update shell/Python loaders and etc/deploy scripts to resolve named configs cleanly while keeping legacy fallback behavior. Align k3s Ansible tasks, docs, and regression coverage with the new configuration layout.

Co-authored-by: Junie <junie@jetbrains.com>
2026-04-11 22:20:45 -07:00
chrisfu
4a8d9cc90d feat: full GKE/prod deployment pipeline from UI to Artifact Registry
## GCP / Cluster Environment Screen
- Auto-populate Cloud tab from conf/prod/gcp.cfg on screen open (org_id,
  billing_account, billing_project, project_id)
- gcloud auth validity checked on screen startup; friendly modal dialog
  streams gcloud auth login output live so user never leaves the app
- Live GKE cluster browser: fetches clusters via gcloud container clusters
  list, displays with checkmark selector, auto-selects saved cluster
- Selecting a cluster runs get-credentials, sets KUBECONFIG/KUBECONTEXT,
  and syncs the region dropdown to the selected cluster's location
- Region dropdown populated live from gcloud compute regions list with
  checkmark on currently selected region; graceful fallback when offline
- New 'GCP Storage' tab with workload->StorageClass mapping (CNPG->premium-rwo,
  Redis/Monitoring->standard-rwo, Garage->garage-hdd) and Fetch from Cluster
- Provider readonly field styled correctly (no solid-black on macOS)
- Stale prole.cfg/conf/prole.cfg symlinks removed; all config I/O now
  resolves env-specific paths via prole_conf.entrypoint_path()

## GKE Autopilot Compatibility (Common Services)
- Synology iSCSI StorageClass and static PVs guarded behind PROLE_MODE!=k8s
  in init_openbao.sh (GKE Autopilot forbids hostPath/iSCSI volumes)
- In-cluster Docker registry (hostPath) skipped in k8s mode; GCP Artifact
  Registry used instead
- Kong renamed knoe-svc-kong in k8s mode; all health-check kubectl calls in
  init_common_services.sh and status_common_services.sh updated accordingly
- DNS endpoints switched from *.prole.org to *.knoe.dev in k8s mode
  (api.knoe.dev, git.knoe.dev, svc.knoe.dev); ingress uses gce class
- New GKE-clean Kong manifests under deploy/opentofu/k8s/manifests/prole/:
  no k3s node affinity, explicit Autopilot resource requests/limits

## Garage S3 Store (GKE)
- New garage-statefulset-gcp.yaml targeting garage-hdd StorageClass
  (pd-standard, avoids SSD_TOTAL_GB quota exhaustion in us-west3)
- New storageclass-gcp-hdd.yaml (pd-standard, Retain, WaitForFirstConsumer)
- GCP StorageClass manifests skipped on re-runs (Autopilot built-ins are
  immutable; skip-if-exists guard added)
- PVC deletion guard extended to cover any storageClass (not just synology)
  so stale claims are cleaned before StatefulSet recreation

## Topology (GKE Autopilot)
- DaemonSet collector skipped in prod mode (forbidden in kube-system by
  GKE Warden); Kubernetes-only node facts path used instead
- All ready GKE nodes assumed cnpg-eligible and monitoring-eligible without
  taint/synology-mount checks (skip_collector + assume_nodes_eligible flags)

## KUBECONFIG / kubectl (k8s mode)
- actions.py: new elif mode==k8s branch sets KUBECONFIG=~/.kube/config
  and injects KUBECONTEXT from prole_cfg_data into script env
- _build_kubectl_cmd falls back to Global.KUBECONTEXT when
  init_cluster.selected_kubectx is empty
- _activate_selected_gke_cluster persists KUBECONFIG/KUBECONTEXT to
  prole_cfg_data and saves prole.cfg immediately after get-credentials

## Database Build Screen (GKE)
- Registry display shows correct Artifact Registry URL
  (<region>-docker.pkg.dev/<project>/<namespace>/knoe-db) in green
- Build+push: gcloud auth configure-docker, auto-creates AR repository
  named after SERVICE_NAMESPACE (e.g. knoe-system) if missing, then
  docker tag + push; falls back to gcr.io if region unavailable
- GCP config loaded from conf/prod/gcp.cfg on every screen entry;
  keys normalised to lowercase so project_id lookup is always consistent

## Config / Namespace persistence
- prole_conf.py activate_environment: symlink creation removed; sets
  CLUSTER_ENV env-var so all subsequent calls resolve correct env directory
- knoe/ui/screens/__init__.py: startup config load uses entrypoint_path()
  instead of hardcoded conf/prole.cfg; seeds SERVICE_NAMESPACE=knoe-system
  for managed envs so Common Services never defaults to 'default'
- cfg.py _save_prole_cfg: saves to env-specific path via entrypoint_path()
- etc/prole_cfg.sh: removed all ln -snf symlink creation

Co-authored-by: Junie <junie@jetbrains.com>
2026-04-04 12:38:16 -07:00
chrisfu
c8505e6976 Replace gcloud auth login with in-TUI email + access token credential form
Remove all browser/device-code auth flows. User now enters their Google
account email (visible) and access token (masked) directly in the ncurses
TUI. Token is passed as CLOUDSDK_AUTH_ACCESS_TOKEN env var to all gcloud
calls — no gcloud auth state is modified. Token email is resolved via
the Google tokeninfo endpoint for display only.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-31 23:35:40 -07:00
chrisfu
fabd1a3463 Fix gcloud auth login output capture swallowing device-code URL
gcloud auth login --no-browser was called via _gcloud_plain which uses
capture_output=True, silently swallowing the device-code URL that the
user needs to complete authentication. Run it directly with no capture.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-31 23:21:19 -07:00
chrisfu
2e517686a8 Add GCP ncurses TUI to knoe/config.py and config.sh launcher
Consolidates the GCP cluster configuration TUI (formerly etc/config.py)
into knoe/config.py as a __main__ entrypoint, resolving the naming
collision with the root-level installer utilities module. Adds config.sh
as a thin shell launcher at the project root. Also excludes .claude/
worktree directories from git and IDE indexing.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-31 23:15:54 -07:00
chrisfu
93f7259722 chore: checkpoint k3s deployment and knoe updates
- update Kong/Supabase ingress and service manifests for k3s

- expand init scripts and runtime config/topology handling

- refresh KNØE UI screens and Supabase deployment/render logic

- include related env helper test adjustments

Co-authored-by: Junie <junie@jetbrains.com>
2026-03-31 23:15:54 -07:00
chrisfu
d02aadae89 Add node management workflow and installer config persistence updates
- add tools/manage-node.sh for k3s node add/remove automation with inventory reconciliation

- include kubectl drain flag compatibility and --ssh-host targeting for uninstall workflows

- persist installer cfg/env values and update deploy/database/cfg UI handling

- refresh related inventory/network artifacts and expand installer/database test coverage

Co-authored-by: Junie <junie@jetbrains.com>
2026-03-29 11:22:33 -07:00
chrisfu
1353882260 Update PROLE_HOME resolution and enhance iSCSI PV handling
- Implement centralized `resolve_prole_home` utility for consistent environment-based `PROLE_HOME` resolution across modules
- Replace hardcoded home paths with `resolve_prole_home`
- Refactor PV management to support iSCSI mounts and node placement from Ansible manifests
- Improve Kubernetes manifest handling to dynamically apply namespaces per document
- Adjust `knoe-db` build context path and related tests
- Add utilities for detecting and applying Ansible-defined node labels and PVs
2026-03-22 02:43:52 -07:00
chrisfu
4ee2b259c9 Checkpoint: rename installer to knoe + harden db build context
- Add build-context helper to copy Docker context safely (ignore runtime data, keep symlinks)

- Update UI and core actions to use ~/.prole/build and shared copy helper

- Add/adjust tests and scripts; introduce knoe ops helpers and update manifests

Co-authored-by: Junie <junie@jetbrains.com>
2026-03-22 01:45:21 -07:00