- Added detailed handling and cleanup for stale retained PVs to prevent quota exhaustion.
- Introduced `check_supabase_retained_pv_blocked` utility for preflight artifact detection and optional auto-cleanup.
- Improved ingress reconciliation for `supabase-kong` and `supabase-studio`.
- Enhanced retry logic with optional destructive namespace resets when the app cluster is unhealthy.
- Added `gitlab_selector_for_deployment` utility to dynamically resolve deployment selectors.
- Updated pod count and name retrieval to use resolved selectors, ensuring accuracy.
- Enhanced logging to include selector details for better debugging and remediation tracking.
- Addressed replica drift detection with CR-based reconciliation and scale-down logic.
- Added settle verification logic with configurable timeout and polling intervals for corrected deployments.
- Introduced new utility function `gitlab_non_terminal_pod_names_for_app` to retrieve non-terminal pod names.
- Improved logging for final deployment state and pod details after remediation actions.
- Added settle verification logic with configurable timeout and polling intervals for corrected deployments.
- Introduced new utility function `gitlab_non_terminal_pod_names_for_app` to retrieve non-terminal pod names.
- Improved logging for final deployment state and pod details after remediation actions.
- Added `_resolve_secretref_value` to handle `secretref://` values for `auth.clientId` and `auth.clientSecret`.
- Extended environment and file-based lookups for secretref resolution.
- Updated tests to validate new secretref resolution logic in config and GitOps workflows.
- Implemented `auth.clientId` and `auth.clientSecret` backfilling from Global section and environment variables.
- Added tests to verify correct hydration of OIDC inputs in configuration defaults.
- Introduced `gitlab_post_remediation_success` flag to streamline corrective action logging.
- Adjusted logic to skip unnecessary workload convergence gates when remediation is successful.
- Introduced `gitlab_post_remediation_success` flag to streamline corrective action logging.
- Adjusted logic to skip unnecessary workload convergence gates when remediation is successful.
Summarize CLAUDE.md updates:
- Document both live GKE clusters as e2-standard-2 x3 in us-west3 and note stale app default in code.
- Call out DB cluster naming mismatch between conf (knoe-dev-cnpg-0) and code default (knoe-cnpg-0).
- Add SSD quota guidance: 300GB fully consumed by CNPG, so non-CNPG PVCs must use standard/pd-standard.
- Refresh reset script reference and quota section details to match current runtime state.
Co-authored-by: Junie <junie@jetbrains.com>
- Added detailed logging for APP and DB contexts in `init_gitlab.sh` and `deploy.sh`.
- Enabled explicit passing of contexts and configuration paths to avoid fallbacks.
- Introduced strict environment resolution logic in `prole_cfg.sh` with fail-safe mechanisms.
- Added skipping logic for aged or bound PVC events in `deploy.sh` to reduce noise in diagnostics.
- Enforced replica target of 1 for specific GitLab workloads to ensure compliance with requirements.
- Added skipping logic for aged or bound PVC events in `deploy.sh` to reduce noise in diagnostics.
- Enforced replica target of 1 for specific GitLab workloads to ensure compliance with requirements.
- Added skipping logic for aged or bound PVC events in `deploy.sh` to reduce noise in diagnostics.
- Enforced replica target of 1 for specific GitLab workloads to ensure compliance with requirements.
- Improved PVC error diagnostics in `deploy.sh` with additional context.
- Added robust StorageClass reconciliation logic, including dynamic creation and parameter validation.
- Updated Helm templates to support advanced StorageClass attributes like `reclaimPolicy` and `volumeBindingMode`.
- Redirected error messages in `deploy.sh` and `render_supabase.py` to `stderr` for better logging clarity.
- Extended allowed prefixes for Supabase StorageClass to include `pd-standard`.
- Added hard validation for Supabase StorageClass configuration in `deploy.sh` and Helm templates.
- Implemented detailed reconciliation logic for GitLab workloads, addressing over-deployment scenarios.
- Enhanced `deploy.sh` with explicit namespace handling and PVC compliance checks.
- Added PVC quota error detection with detailed logging in `deploy.sh`.
- Enhanced ingress readiness checks with additional Supabase ingress resources.
- Updated Helm templates to standardize and always enable ingress for API and Studio.
- Added PVC quota error detection with detailed logging in `deploy.sh`.
- Enhanced ingress readiness checks with additional Supabase ingress resources.
- Updated Helm templates to standardize and always enable ingress for API and Studio.
- Simplified public APP endpoint fallback logic in `deploy.sh`.
- Enhanced timeout handling with detailed Supabase ingress reconciliation diagnostics.
- Removed redundant ingress class override in Helm template for Supabase.
- Standardized naming from "Prole" to "Knoe" in README, scripts, and deployment files for consistency.
- Refined ingress configuration to adjust class and annotations based on mode.
- Enhanced deploy.sh to handle public APP endpoint status for specific hosts.
- Standardized naming from "Prole" to "Knoe" in README, scripts, and deployment files for consistency.
- Refined ingress configuration to adjust class and annotations based on mode.
- Enhanced deploy.sh to handle public APP endpoint status for specific hosts.
- Resolved context mismatches by updating DB references to APP where appropriate.
- Updated Supabase deployment to ensure proper handling of cross-cluster DB hosts.
- Simplified ingress logic by setting `split_frontdoor_to_db` to `false` for APP clusters lacking GCE Ingress support.
- Resolved context mismatches by updating DB references to APP where appropriate.
- Updated Supabase deployment to ensure proper handling of cross-cluster DB hosts.
- Simplified ingress logic by setting `split_frontdoor_to_db` to `false` for APP clusters lacking GCE Ingress support.
- Updated `init_kong.sh` to handle `include_gitea_host` flag for improved flexibility in Gitea ingress setup.
- Added comprehensive post-deploy endpoint summary in `deploy.sh` to display public services and external IPs.
- Enhanced kubectl command handling with role-specific context logic for better split-cluster support.
- Improved DB context logging across CNPG operations and milestones.
- Updated GKE configuration with new contexts and Grafana hostname.
- Added automatic persistence disabling when deployment is disabled.
- Enhanced readiness checks with support for selectors and detailed status logging.
- Improved retry logic to avoid unnecessary namespace resets for healthy app clusters.
- Refined helm deployment flow with clearer readiness and health diagnostics.
- Introduced diagnostics for MinIO failures due to file access issues, with suggested fixes for pod security settings.
- Added podSecurityContext and securityContext configurations to enforce non-root execution for MinIO.
- Updated resource requests/limits for MinIO pods to ensure optimal resource usage.
- Standardized file path references in manifest-summary.json to relative paths.
- Added detailed validation of Sidekiq concurrency, resource requests, and limits.
- Introduced live state comparison for Sidekiq resource configuration.
- Improved logging to track desired and live Sidekiq states for better diagnostics.
- Added split-cluster detection with detailed app and DB context logging.
- Introduced robust DB connectivity checks for DNS resolution and TCP probes.
- Enabled automated cross-cluster DB ILB service creation and validation.
- Improved logging for DB migration failures with enhanced diagnostics and connectivity hints.
- Added split-cluster detection with detailed app and DB context logging.
- Introduced robust DB connectivity checks for DNS resolution and TCP probes.
- Enabled automated cross-cluster DB ILB service creation and validation.
- Improved logging for DB migration failures with enhanced diagnostics and connectivity hints.
- Introduced `get_gitlab_migrations_diagnostics` for detailed migrations pod diagnostics.
- Enhanced active migration failure detection with detailed logs and diagnostics.
- Improved job context logging with pod restart, status, and waiting reason details.
- Refined stale job repair logic by incorporating diagnostics and failure insights.
- Introduced `get_gitlab_migrations_diagnostics` for detailed migrations pod diagnostics.
- Enhanced active migration failure detection with detailed logs and diagnostics.
- Improved job context logging with pod restart, status, and waiting reason details.
- Refined stale job repair logic by incorporating diagnostics and failure insights.
- Added detailed context logging for GitLab blockers, including failed migrations and Gitaly resources.
- Introduced automatic repair for stalled migration jobs by deleting stale jobs and pods.
- Improved Gitaly resource validation with enhanced logging for PVCs and StatefulSet configurations.
- Extended PVC phase handling with detailed storageClass descriptions for better troubleshooting.
- Improved detection and logging of mismatched storageClass across CR, StatefulSet, and PVCs.
- Added auto-repair mechanisms for inconsistencies in Gitaly storageClass configuration.
- Enhanced state validation with detailed logs for better troubleshooting.
- Introduced a grace period and explicit replacement of legacy StatefulSet after expiry.
- Enhanced logging to provide clearer details on convergence progress and outcomes.
- Added handling for operator recreation and state validation during legacy repairs.
- Ensured robust detection and resolution of legacy nodeSelector and storageClass issues.
- Added logic to wait for Gitaly StatefulSet corrections after autoclean actions.
- Implemented live state validation for nodeSelector and storageClass updates.
- Enhanced logging and error handling for legacy storage repairs in k8s mode.
- Updated PV and PVC handling to address legacy resource cleanup.
- Added logging to show Kubernetes context and service inspection details for Redis.
- Improved validation by checking for service endpoints and readiness.
- Enabled IP address bypass for external host DNS checks.
- Added logging to show Kubernetes context and service inspection details for Redis.
- Improved validation by checking for service endpoints and readiness.
- Enabled IP address bypass for external host DNS checks.
- Added fallback logic to resolve Garage ILB hostname if IP is unavailable.
- Improved handling of `self.inputs` with prioritized merging for better automation.
- Updated config persistence to use `write_config` for milestones and repairs.
- Extended Garage ILB IP retry logic to 5 minutes for GKE provisioning delays.
- Changed ILB deployment context to DB cluster (`knoe-cnpg-0`) for GitLab object storage setup.
- Renamed service to `garage-s3-ilb` and updated related references.
- Enhanced logging for Garage ILB IP resolution with retry and error handling.
- Persisted resolved ILB endpoint to config files for improved reliability.
- Refined split-cluster Garage deployment workflows with clearer context management and debugging.
