Filed in response to the 2026-04-28 14:00 UTC backup outage. An
`install.sh --mode k3d` run with the shell pointed at GKE silently
overwrote the GKE cluster's GCS-backed ObjectStore + ScheduledBackup
with k3d-mode defaults; Garage filled up and CNPG backups failed for
hours before the next manual check. The class of bug is "config says
target cluster A, shell context says target cluster B, installer
proceeds against B without warning."
New shared bash helper at etc/preflight_kubecontext.sh with two
functions:
- verify_kubecontext_matches_config <cfg-path>
Strict gate. Reads [Global] APP_CLUSTER_KUBECONTEXT from the
config and exits 1 if `kubectl config current-context` differs.
Skipped silently when the config has no baked APP_CLUSTER_KUBECONTEXT
(e.g. fresh k3d.cfg) or when there's no live current-context.
- print_kubecontext_notice
Informational. Prints what's about to be inherited so the user
can abort before the TUI launches if it looks wrong. Never fails.
Wiring:
- deploy.sh sources the helper and calls the strict gate against
${PROLE_DEPLOY_CFG:-conf/gke.cfg} before invoking Python.
Unattended path -> hard refusal on mismatch.
- install.sh sources the helper and calls the informational notice
(gated on not-`--min`) right after entering the local-checkout
branch. The TUI is interactive, so the strict mode-aware gate is
a follow-up once the welcome screen records a mode in
state.inputs.
Bypass for deliberate cross-cluster maintenance:
KNOE_SKIP_KUBECONTEXT_GUARD=true ./deploy.sh
End-to-end verified:
- deploy.sh with current=cnpg-0, gke.cfg=app-0 -> exit 1, clear msg
- deploy.sh with KNOE_SKIP_...=true -> bypasses, prints
"skipping check"
- install.sh --min -> notice skipped
- install.sh (no flag) and install.sh --silent -> notice printed
Doc updates:
- CLAUDE.md §"Env-contamination warning" rewritten to describe the
live guard (was a forward-looking TODO).
- CLAUDE.md drift table row R4 removed; "Closed 2026-05-01" line added.
- docs/TODO.md queue item #1 archived to Done; R4 dropped from the
reality-vs-intent table. Queue numbering retained (no #1 placeholder)
so the docs/plans/junie/<NN>-...md filenames still match.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Added pre-apply validation for ManagedCertificate and FrontendConfig objects to ensure existence before ingress application.
- Improved GKE ingress diagnostics with annotations and resource state checks for pre- and post-apply phases.
- Introduced patch-only handling for annotation drifts to avoid unnecessary ingress replacements.
- Enhanced HTTPS probing logic with GCLB convergence checks for managed certificate states.
- Updated `deploy.sh`, `init_kong.sh`, and `init_gitlab.sh` with detailed ingress dependency validation and reconciliation steps.
- Introduced dependency validation for ManagedCertificate and FrontendConfig annotations in Supabase ingresses.
- Enhanced `deploy.sh` with public ingress TLS resource application and improved error diagnostics.
- Updated `render_supabase.py` to generate GKE TLS manifests and log rendered resources.
- Refined Supabase deployment flow to include detailed ingress dependency validation and reconciliation.
- Improved handling of GKE preload TLS annotations by ignoring `SERVICE_PRE_SHARED_CERT` in k8s mode with managed TLS resources.
- Added drift detection for ingress class, ManagedCertificate, and FrontendConfig annotations, with forced replacement for GCLB binding consistency.
- Enhanced TLS diagnostics with detailed path state, failure reasons, and HTTPS probe handling for GKE configurations.
- Updated tests and scripts to validate GKE-specific behavior and ingress rendering.
- Updated `deploy.sh` and `init_kong.sh` to support GKE ManagedCertificate and FrontendConfig reconciliation in k8s mode with GCE ingress class.
- Enhanced TLS path diagnostics to distinguish between missing, attached-but-not-serving, and actively serving configurations.
- Added HTTPS probing for GKE-managed public ingress paths and validation for managed certificate statuses.
- Updated `render_supabase.py` to generate and attach TLS annotations for Supabase API and Studio ingresses in k8s mode.
- Added tests to validate TLS path handling, probing, and manifest generation for GKE deployments.
- Enhanced `deploy.sh` to include detailed storage class diagnostics for blocking PVCs (e.g., provisioner, type, binding mode).
- Added `validate_supabase_rendered_pvc_storage` to ensure rendered PVC configurations align with expected storage classes.
- Improved event parsing logic to prioritize recent PVC errors and resolve storage class details.
- Updated deployment flow to handle Supabase Studio ingress reconciliation errors if deployment is missing or replicas are zero.
- Introduced `GITLAB_FALLBACK_INGRESS_NAME` for better flexibility in managing fallback ingress names.
- Added logic to clean up both current and legacy fallback ingresses (`gitlab-frontdoor-ingress`, `gitlab-kong-ingress`).
- Enhanced diagnostics for ingress host conflicts and TLS path validation.
- Updated `deploy.sh` to improve runtime configuration resolution and host diagnostics.
- Refined tests to ensure correct fallback ingress handling and conflict resolution.
- Replaced `is_gke_standard_storage_equivalent` with `gitlab_storage_class_matches_expected` for stricter logic and better readability.
- Added utilities for diagnosing and cleaning up PVC/PV and StatefulSet template storage class mismatches.
- Introduced `enforce_gitlab_storage_class_target` to automate configuration enforcement and diagnostics.
- Enhanced logging and failure remediation for storage class mismatches in GitLab deployments.
- Added `supabase_collect_stale_retained_pv_info` for clearer separation of logic and improved reusability.
- Enhanced zone detection for retained PVs using regex-based fallback.
- Updated cleanup logic to auto-enable `SUPABASE_AUTO_CLEAN_RETAINED_PVS` and streamline handling of GCE disk deletion.
- Introduced verification and retry mechanism to ensure stale PV artifacts are fully removed.
- Improved error handling for remaining GCE disk cleanup with detailed remediation instructions.
- Added detailed handling and cleanup for stale retained PVs to prevent quota exhaustion.
- Introduced `check_supabase_retained_pv_blocked` utility for preflight artifact detection and optional auto-cleanup.
- Improved ingress reconciliation for `supabase-kong` and `supabase-studio`.
- Enhanced retry logic with optional destructive namespace resets when the app cluster is unhealthy.
Summarize CLAUDE.md updates:
- Document both live GKE clusters as e2-standard-2 x3 in us-west3 and note stale app default in code.
- Call out DB cluster naming mismatch between conf (knoe-dev-cnpg-0) and code default (knoe-cnpg-0).
- Add SSD quota guidance: 300GB fully consumed by CNPG, so non-CNPG PVCs must use standard/pd-standard.
- Refresh reset script reference and quota section details to match current runtime state.
Co-authored-by: Junie <junie@jetbrains.com>
- Added detailed logging for APP and DB contexts in `init_gitlab.sh` and `deploy.sh`.
- Enabled explicit passing of contexts and configuration paths to avoid fallbacks.
- Introduced strict environment resolution logic in `prole_cfg.sh` with fail-safe mechanisms.
- Added skipping logic for aged or bound PVC events in `deploy.sh` to reduce noise in diagnostics.
- Enforced replica target of 1 for specific GitLab workloads to ensure compliance with requirements.
- Added skipping logic for aged or bound PVC events in `deploy.sh` to reduce noise in diagnostics.
- Enforced replica target of 1 for specific GitLab workloads to ensure compliance with requirements.
- Added skipping logic for aged or bound PVC events in `deploy.sh` to reduce noise in diagnostics.
- Enforced replica target of 1 for specific GitLab workloads to ensure compliance with requirements.
- Redirected error messages in `deploy.sh` and `render_supabase.py` to `stderr` for better logging clarity.
- Extended allowed prefixes for Supabase StorageClass to include `pd-standard`.
- Added hard validation for Supabase StorageClass configuration in `deploy.sh` and Helm templates.
- Implemented detailed reconciliation logic for GitLab workloads, addressing over-deployment scenarios.
- Enhanced `deploy.sh` with explicit namespace handling and PVC compliance checks.
- Added PVC quota error detection with detailed logging in `deploy.sh`.
- Enhanced ingress readiness checks with additional Supabase ingress resources.
- Updated Helm templates to standardize and always enable ingress for API and Studio.
- Added PVC quota error detection with detailed logging in `deploy.sh`.
- Enhanced ingress readiness checks with additional Supabase ingress resources.
- Updated Helm templates to standardize and always enable ingress for API and Studio.
- Simplified public APP endpoint fallback logic in `deploy.sh`.
- Enhanced timeout handling with detailed Supabase ingress reconciliation diagnostics.
- Removed redundant ingress class override in Helm template for Supabase.
- Standardized naming from "Prole" to "Knoe" in README, scripts, and deployment files for consistency.
- Refined ingress configuration to adjust class and annotations based on mode.
- Enhanced deploy.sh to handle public APP endpoint status for specific hosts.
- Resolved context mismatches by updating DB references to APP where appropriate.
- Updated Supabase deployment to ensure proper handling of cross-cluster DB hosts.
- Simplified ingress logic by setting `split_frontdoor_to_db` to `false` for APP clusters lacking GCE Ingress support.
- Updated `init_kong.sh` to handle `include_gitea_host` flag for improved flexibility in Gitea ingress setup.
- Added comprehensive post-deploy endpoint summary in `deploy.sh` to display public services and external IPs.
- Enhanced kubectl command handling with role-specific context logic for better split-cluster support.
- Improved DB context logging across CNPG operations and milestones.
- Updated GKE configuration with new contexts and Grafana hostname.
- Updated default paths, namespaces, and cluster names for k3d/dev configuration.
- Added `skip_names` set to exclude `prod.cfg` and `gcp.cfg` from override processing.
- Refined init parameters for Supabase, Kubernetes, and database deployments to align with dev-specific settings.
- Enhanced test coverage for excluded config files (`prod.cfg`, `gcp.cfg`) within `test_prole_conf.py`.
