chrisfu/prole
1
0
Fork 0
mirror of https://github.com/dredx/prole.git synced 2026-09-24 19:14:33 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity
1575f4f0b3
prole/docs/deploy_gke.log
chrisfu b8a55888be checkpoint: update GKE deployment docs and infra changes 
Summarize CLAUDE.md updates:

- Document both live GKE clusters as e2-standard-2 x3 in us-west3 and note stale app default in code.

- Call out DB cluster naming mismatch between conf (knoe-dev-cnpg-0) and code default (knoe-cnpg-0).

- Add SSD quota guidance: 300GB fully consumed by CNPG, so non-CNPG PVCs must use standard/pd-standard.

- Refresh reset script reference and quota section details to match current runtime state.

Co-authored-by: Junie <junie@jetbrains.com>
2026-04-18 07:18:10 -07:00

441 lines
28 KiB
Plaintext
Raw Blame History

chrisfu@myrddin:~/dev/prole $ ./deploy.sh -c conf/gke.cfg
[CONFIG] Wrote conf/gke.cfg
[CONFIG] Using conf/gke.cfg
[10%] Checking dependencies...
[10%] Checking python...
[16%] Checking ansible...
[21%] Checking kubectl...
[27%] Checking kubectx...
[33%] Checking gcloud...
[39%] Checking Docker...
[44%] Checking gke-gcloud-auth-plugin...
[100%] All dependencies installed
[50%] Setting up environment...
[100%] Environment setup complete
[50%] Resolving secrets...
[100%] Secret resolution complete
[50%] Preparing database configuration...
[100%] Database creation preparation complete
[OPENBAO] Applying /home/chrisfu/dev/prole/k8s/openbao/deployment.yaml
[OPENBAO] Applying /home/chrisfu/dev/prole/k8s/openbao/kerberos-configmap.yaml
[100%] Cluster lifecycle management complete
[0%] Running common services (python owners)...
[17%] Initializing CloudNative-PG...
[40%] Running init_cnpg_backup.sh...
Checking Garage readiness ...
Garage layout version 1 is applied and ready.
Ensuring Garage bucket and access key for backups ...
2026-04-13T19:26:29.011543Z INFO garage_net::netapp: Connected to 34.118.234.23:3901, negotiating handshake...
2026-04-13T19:26:29.053341Z INFO garage_net::netapp: Connection established to e1be76b655e73136
New permissions for GK96d5ffe8b11f3a3dc7a09a76 on knoe-db-backups: read true, write true, owner true.
Creating/updating Kubernetes secret 'knoe-db-barman-s3' ...
secret/knoe-db-barman-s3 unchanged
Applying Barman Cloud ObjectStore 'knoe-db-barman-objectstore' ...
objectstore.barmancloud.cnpg.io/knoe-db-barman-objectstore unchanged
Waiting for ObjectStore 'knoe-db-barman-objectstore' to be Ready (timeout: 180s)...
ObjectStore 'knoe-db-barman-objectstore' is Ready.
Ensuring ScheduledBackup 'knoe-db-scheduled-backup' ...
scheduledbackup.postgresql.cnpg.io/knoe-db-scheduled-backup unchanged
Waiting for CNPG webhook service endpoints to be ready (timeout: 300s)...
Triggering full backup knoe-db-backup-20260413122642 ...
backup.postgresql.cnpg.io/knoe-db-backup-20260413122642 created
CNPG backup status for cluster 'knoe-db' (namespace: 'knoe-db-0')
Cluster spec.backup: MISSING
ContinuousArchiving condition:
- status=True reason=ContinuousArchivingSuccess message=Continuous archiving is working
ScheduledBackup resources:
NAME AGE CLUSTER LAST BACKUP
knoe-db-scheduled-backup 3d5h knoe-db 23m
Backup resources:
knoe-db-backup-20260410065209 3d5h knoe-db plugin failed requested plugin is not available: barman-cloud.cloudnative-pg.io
knoe-db-backup-20260410065222 3d5h knoe-db plugin failed requested plugin is not available: barman-cloud.cloudnative-pg.io
knoe-db-backup-20260410065230 3d5h knoe-db plugin failed requested plugin is not available: barman-cloud.cloudnative-pg.io
knoe-db-backup-20260410065237 3d5h knoe-db plugin failed requested plugin is not available: barman-cloud.cloudnative-pg.io
knoe-db-backup-20260410065245 3d5h knoe-db plugin failed requested plugin is not available: barman-cloud.cloudnative-pg.io
knoe-db-backup-20260410065253 3d5h knoe-db plugin completed
knoe-db-backup-20260410065849 3d5h knoe-db plugin completed
knoe-db-backup-20260410091200 3d3h knoe-db plugin completed
knoe-db-backup-20260410092247 3d3h knoe-db plugin completed
knoe-db-backup-20260410094119 3d2h knoe-db plugin completed
knoe-db-backup-20260410095332 3d2h knoe-db plugin completed
knoe-db-backup-20260410111426 3d1h knoe-db plugin completed
knoe-db-backup-20260410112620 3d1h knoe-db plugin completed
knoe-db-backup-20260410114805 3d knoe-db plugin completed
knoe-db-backup-20260410120140 3d knoe-db plugin completed
knoe-db-backup-20260410121003 3d knoe-db plugin completed
knoe-db-backup-20260410122407 3d knoe-db plugin completed
knoe-db-backup-20260411195730 40h knoe-db plugin completed
knoe-db-backup-20260411200312 40h knoe-db plugin completed
knoe-db-backup-20260411201013 40h knoe-db plugin completed
knoe-db-backup-20260412184731 17h knoe-db plugin completed
knoe-db-backup-20260412201640 16h knoe-db plugin completed
knoe-db-backup-20260412212354 15h knoe-db plugin completed
knoe-db-backup-20260412214440 14h knoe-db plugin completed
knoe-db-backup-20260412214656 14h knoe-db plugin completed
knoe-db-backup-20260412220633 14h knoe-db plugin completed
knoe-db-backup-20260412224914 13h knoe-db plugin completed
knoe-db-backup-20260412225948 13h knoe-db plugin completed
knoe-db-backup-20260412232601 13h knoe-db plugin completed
knoe-db-backup-20260412232938 12h knoe-db plugin completed
knoe-db-backup-20260413000009 12h knoe-db plugin completed
knoe-db-backup-20260413001240 12h knoe-db plugin completed
knoe-db-backup-20260413005602 11h knoe-db plugin completed
knoe-db-backup-20260413010245 11h knoe-db plugin completed
knoe-db-backup-20260413011235 11h knoe-db plugin completed
knoe-db-backup-20260413105804 88m knoe-db plugin completed
knoe-db-backup-20260413110557 80m knoe-db plugin completed
knoe-db-backup-20260413114901 37m knoe-db plugin completed
knoe-db-backup-20260413122642 2s knoe-db plugin started
knoe-db-scheduled-backup-20260410135209 3d5h knoe-db plugin failed requested plugin is not available: barman-cloud.cloudnative-pg.io
knoe-db-scheduled-backup-20260410140300 3d5h knoe-db plugin completed
knoe-db-scheduled-backup-20260410150300 3d4h knoe-db plugin completed
knoe-db-scheduled-backup-20260410160300 3d3h knoe-db plugin completed
knoe-db-scheduled-backup-20260410170300 3d2h knoe-db plugin completed
knoe-db-scheduled-backup-20260410180300 3d1h knoe-db plugin completed
knoe-db-scheduled-backup-20260410190300 3d knoe-db plugin completed
knoe-db-scheduled-backup-20260410200300 2d23h knoe-db plugin completed
knoe-db-scheduled-backup-20260410210300 2d22h knoe-db plugin completed
knoe-db-scheduled-backup-20260410220300 2d21h knoe-db plugin completed
knoe-db-scheduled-backup-20260410230300 2d20h knoe-db plugin completed
knoe-db-scheduled-backup-20260411000300 2d19h knoe-db plugin completed
knoe-db-scheduled-backup-20260411010300 2d18h knoe-db plugin completed
knoe-db-scheduled-backup-20260411020300 2d17h knoe-db plugin completed
knoe-db-scheduled-backup-20260411030300 2d16h knoe-db plugin completed
knoe-db-scheduled-backup-20260411040300 2d15h knoe-db plugin completed
knoe-db-scheduled-backup-20260411050300 2d14h knoe-db plugin completed
knoe-db-scheduled-backup-20260411060300 2d13h knoe-db plugin completed
knoe-db-scheduled-backup-20260411070300 2d12h knoe-db plugin completed
knoe-db-scheduled-backup-20260411080300 2d11h knoe-db plugin completed
knoe-db-scheduled-backup-20260411090300 2d10h knoe-db plugin completed
knoe-db-scheduled-backup-20260411100300 2d9h knoe-db plugin completed
knoe-db-scheduled-backup-20260411110300 2d8h knoe-db plugin completed
knoe-db-scheduled-backup-20260411120300 2d7h knoe-db plugin completed
knoe-db-scheduled-backup-20260411130300 2d6h knoe-db plugin completed
knoe-db-scheduled-backup-20260411140300 2d5h knoe-db plugin completed
knoe-db-scheduled-backup-20260411150300 2d4h knoe-db plugin completed
knoe-db-scheduled-backup-20260411160300 2d3h knoe-db plugin completed
knoe-db-scheduled-backup-20260411170300 2d2h knoe-db plugin completed
knoe-db-scheduled-backup-20260411180300 2d1h knoe-db plugin completed
knoe-db-scheduled-backup-20260411190300 2d knoe-db plugin completed
knoe-db-scheduled-backup-20260411200300 47h knoe-db plugin completed
knoe-db-scheduled-backup-20260411210300 46h knoe-db plugin completed
knoe-db-scheduled-backup-20260411220300 45h knoe-db plugin completed
knoe-db-scheduled-backup-20260411230300 44h knoe-db plugin completed
knoe-db-scheduled-backup-20260412000300 43h knoe-db plugin completed
knoe-db-scheduled-backup-20260412010300 42h knoe-db plugin completed
knoe-db-scheduled-backup-20260412020300 41h knoe-db plugin completed
knoe-db-scheduled-backup-20260412030300 40h knoe-db plugin completed
knoe-db-scheduled-backup-20260412040300 39h knoe-db plugin completed
knoe-db-scheduled-backup-20260412050300 38h knoe-db plugin completed
knoe-db-scheduled-backup-20260412060300 37h knoe-db plugin completed
knoe-db-scheduled-backup-20260412070300 36h knoe-db plugin completed
knoe-db-scheduled-backup-20260412080300 35h knoe-db plugin completed
knoe-db-scheduled-backup-20260412090300 34h knoe-db plugin completed
knoe-db-scheduled-backup-20260412100300 33h knoe-db plugin completed
knoe-db-scheduled-backup-20260412110300 32h knoe-db plugin completed
knoe-db-scheduled-backup-20260412120300 31h knoe-db plugin completed
knoe-db-scheduled-backup-20260412130300 30h knoe-db plugin completed
knoe-db-scheduled-backup-20260412140300 29h knoe-db plugin completed
knoe-db-scheduled-backup-20260412150300 28h knoe-db plugin completed
knoe-db-scheduled-backup-20260412160300 27h knoe-db plugin completed
knoe-db-scheduled-backup-20260412170300 26h knoe-db plugin completed
knoe-db-scheduled-backup-20260412180300 25h knoe-db plugin completed
knoe-db-scheduled-backup-20260412190300 24h knoe-db plugin completed
knoe-db-scheduled-backup-20260412200300 23h knoe-db plugin completed
knoe-db-scheduled-backup-20260412210300 22h knoe-db plugin completed
knoe-db-scheduled-backup-20260412220300 21h knoe-db plugin completed
knoe-db-scheduled-backup-20260412230300 20h knoe-db plugin completed
knoe-db-scheduled-backup-20260413000300 19h knoe-db plugin completed
knoe-db-scheduled-backup-20260413010300 18h knoe-db plugin completed
knoe-db-scheduled-backup-20260413020300 17h knoe-db plugin completed
knoe-db-scheduled-backup-20260413030300 16h knoe-db plugin completed
knoe-db-scheduled-backup-20260413040300 15h knoe-db plugin completed
knoe-db-scheduled-backup-20260413050300 14h knoe-db plugin completed
knoe-db-scheduled-backup-20260413060300 13h knoe-db plugin completed
knoe-db-scheduled-backup-20260413070300 12h knoe-db plugin completed
knoe-db-scheduled-backup-20260413080300 11h knoe-db plugin completed
knoe-db-scheduled-backup-20260413090300 10h knoe-db plugin completed
knoe-db-scheduled-backup-20260413100300 9h knoe-db plugin completed
knoe-db-scheduled-backup-20260413110300 8h knoe-db plugin completed
knoe-db-scheduled-backup-20260413120300 7h23m knoe-db plugin completed
knoe-db-scheduled-backup-20260413130300 6h23m knoe-db plugin completed
knoe-db-scheduled-backup-20260413140300 5h23m knoe-db plugin completed
knoe-db-scheduled-backup-20260413150300 4h23m knoe-db plugin completed
knoe-db-scheduled-backup-20260413160300 3h23m knoe-db plugin completed
knoe-db-scheduled-backup-20260413170300 143m knoe-db plugin completed
knoe-db-scheduled-backup-20260413180300 83m knoe-db plugin completed
knoe-db-scheduled-backup-20260413190300 23m knoe-db plugin completed
Backup status: OK (at least one successful base backup is available).
[60%] Running init_kong.sh...
Creating/updating Kong declarative config 'knoe-svc-kong-config' in namespace 'knoe-system' ...
configmap/knoe-svc-kong-config unchanged
ConfigMap 'knoe-svc-kong-config' ready.
Applying service Ingress for host 'svc.knoe.dev' -> knoe-svc-kong:8000 (namespace=knoe-system) ...
ingress.networking.k8s.io/svc-knoe-ingress unchanged
Deploying knoe-svc-kong to namespace 'knoe-system' ...
deployment.apps/knoe-svc-kong unchanged
service/knoe-svc-kong unchanged
Waiting for knoe-svc-kong rollout ...
deployment "knoe-svc-kong" successfully rolled out
knoe-svc-kong config unchanged; skipping rollout restart.
knoe-svc-kong deployed successfully.
[80%] Running init_nginx_ingress.sh...
[INFO] nginx ingress host-port mappings are only applied for k3d (mode=k8s).
[83%] Running monitoring (python owner)...
[100%] Initialization scripts complete
[100%] Kerberos disabled; skipping
[40%] Deploying GitLab...
[INFO] GitLab database 'gitlabhq_production' prepared in knoe-db (ns=knoe-db-0, ctx=gke_plenary-truck-485623-p7_us-west3_knoe-dev-cnpg-0).
[INFO] gitlab-db-password secret applied in namespace 'gitlab'.
[INFO] Adding/updating GitLab Operator Helm repo...
"gitlab-operator" already exists with the same configuration, skipping
Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "gitlab-operator" chart repository
Update Complete. ⎈Happy Helming!⎈
[INFO] Installing GitLab Operator (release=gitlab-operator, ns=gitlab)...
Release "gitlab-operator" has been upgraded. Happy Helming!
NAME: gitlab-operator
LAST DEPLOYED: Mon Apr 13 12:28:18 2026
NAMESPACE: gitlab
STATUS: deployed
REVISION: 13
TEST SUITE: None
[INFO] GitLab Operator ready.
[INFO] Auto-detecting latest GitLab chart version...
"gitlab" already exists with the same configuration, skipping
Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "gitlab" chart repository
Update Complete. ⎈Happy Helming!⎈
[INFO] Auto-detected GitLab chart version: 9.10.3
[INFO] Applying gitlab-google-oidc secret (issuer=https://api.knoe.dev/auth) ...
[WARN] Skipping hostPath jemalloc optimization: Detected GKE/GCE node metadata (providerID/labels).
[INFO] HostPath jemalloc optimization disabled for this environment.
[INFO] Skipping legacy Synology/local-PV gitaly storage prep in k8s mode; using dynamic StorageClass 'standard-rwo'.
[INFO] Configuring Garage S3 (knoe-system) as GitLab object storage...
[INFO] Garage admin context: gke_plenary-truck-485623-p7_us-west3_knoe-dev-cnpg-0
[INFO] Garage namespace: knoe-system
[INFO] GitLab object storage S3 endpoint: http://<private-db-cluster-garage-endpoint>:3900
[INFO] Garage key 'gitlab-s3' already exists (id=GK9c13252e4b2dc0cbd1e6c31f); fetching info...
[INFO] Garage bucket 'registry' already exists; reusing existing bucket.
[INFO] Garage bucket 'gitlab-artifacts-storage' already exists; reusing existing bucket.
[INFO] Garage bucket 'gitlab-lfs-storage' already exists; reusing existing bucket.
[INFO] Garage bucket 'gitlab-uploads-storage' already exists; reusing existing bucket.
[INFO] Garage bucket 'gitlab-packages-storage' already exists; reusing existing bucket.
[INFO] Garage bucket 'gitlab-dependency-proxy-storage' already exists; reusing existing bucket.
[INFO] Garage bucket 'gitlab-terraform-state' already exists; reusing existing bucket.
[INFO] Garage bucket 'gitlab-ci-secure-files' already exists; reusing existing bucket.
[INFO] Garage buckets provisioned for GitLab.
[INFO] gitlab-object-storage secret applied.
[INFO] gitlab-registry-storage secret applied.
[INFO] Pre-creating gitlab-app-nonroot ServiceAccount (required by chart v9+)...
serviceaccount/gitlab-app-nonroot unchanged
[INFO] Rendering GitLab values without hostPath jemalloc mounts/LD_PRELOAD.
[INFO] Applying GitLab CR (hosts=git.knoe.dev, db=gitlabhq_production@knoe-db-rw.knoe-db-0.svc.cluster.local)...
gitlab.apps.gitlab.com/gitlab unchanged
[INFO] Skipping ARM64 configure patch — GitLab v18 CNG images are multi-arch natively.
[INFO] GitLab CR applied — operator is reconciling (this may take 10-20 minutes).
[INFO] Monitor progress: kubectl -n gitlab get gitlab gitlab -w
[INFO] Watch pods: kubectl -n gitlab get pods -w
[INFO] Waiting up to 1200s for GitLab CR to reach Ready status...
[WARN] Timed out waiting for GitLab CR condition=Available.
[WARN] The deployment may still be in progress. Check: kubectl -n gitlab describe gitlab gitlab
[INFO] Ensuring ingress (gce) for hosts=git.knoe.dev -> gitlab-webservice-default:8181 ...
[INFO] Operator-managed ingress already owns GitLab host/path in namespace 'gitlab': git.knoe.dev/ already owned by gitlab/gitlab-webservice-default (backend=gitlab-webservice-default)
[INFO] Skipping fallback ingress creation for GitLab release 'gitlab'.
[INFO] --- Registry migration: registry:2 → gitlab-registry ---
Registry migration: registry.knoe-system.svc.cluster.local:5000 → gitlab-registry.gitlab.svc.cluster.local:5000
INFO: registry:2 not found in namespace 'knoe-system' — nothing to migrate.
[INFO] Done.
[INFO] GitLab will be reachable at hosts=git.knoe.dev once pods are Running.
[INFO] Initial root password: kubectl -n gitlab get secret gitlab-gitlab-initial-root-password -o jsonpath='{.data.password}' | base64 -d
[100%] GitLab deployed
[100%] Supabase image preload not required for mode 'k8s'
==> Switching kubectl context to APP cluster 'gke_plenary-truck-485623-p7_us-west3_knoe-dev-0' for Supabase deployment...
==> Resetting 'supabase' namespace...
==> Creating 'supabase' namespace...
namespace/supabase created
==> Ensuring cross-cluster Postgres ILB service 'knoe-db-rw-ilb' in namespace 'knoe-db-0'...
==> Using cross-cluster Supabase DB host: 10.180.15.236
==> Rendering Supabase Helm values/manifests...
==> Ensuring Supabase roles, schemas and databases exist in knoe-db...
NOTICE: table "migrations" does not exist, skipping
NOTICE: table "migrations" does not exist, skipping
NOTICE: role "authenticator" has already been granted membership in role "anon" by role "postgres"
NOTICE: role "authenticator" has already been granted membership in role "authenticated" by role "postgres"
NOTICE: role "authenticator" has already been granted membership in role "service_role" by role "postgres"
NOTICE: role "authenticator" has already been granted membership in role "supabase_admin" by role "postgres"
NOTICE: schema "knoe" already exists, skipping
NOTICE: schema "auth" already exists, skipping
NOTICE: schema "storage" already exists, skipping
NOTICE: schema "graphql_public" already exists, skipping
NOTICE: schema "_realtime" already exists, skipping
DROP TABLE
DROP TABLE
DO
DO
DO
DO
DO
DO
DO
DO
DO
DO
DO
DO
ALTER ROLE
ALTER ROLE
ALTER ROLE
ALTER ROLE
ALTER ROLE
ALTER ROLE
ALTER ROLE
ALTER ROLE
ALTER ROLE
ALTER ROLE
ALTER ROLE
GRANT
GRANT
GRANT ROLE
GRANT ROLE
GRANT ROLE
GRANT ROLE
CREATE SCHEMA
CREATE SCHEMA
CREATE SCHEMA
GRANT
CREATE SCHEMA
CREATE SCHEMA
ALTER SCHEMA
GRANT
GRANT
GRANT
GRANT
GRANT
ALTER DEFAULT PRIVILEGES
GRANT
GRANT
GRANT
GRANT
DO
GRANT
DO
NOTICE: schema "knoe" already exists, skipping
NOTICE: schema "_supavisor" already exists, skipping
NOTICE: schema "_analytics" already exists, skipping
CREATE SCHEMA
GRANT
CREATE SCHEMA
ALTER SCHEMA
CREATE SCHEMA
ALTER SCHEMA
==> Supabase database roles and schemas ready.
==> Using storage class 'supabase-standard-wffc' on GKE k8s mode.
==> Installing Supabase via Helm into namespace 'supabase' (attempt 1/2)...
Release "supabase" does not exist. Installing it now.
NAME: supabase
LAST DEPLOYED: Mon Apr 13 12:49:22 2026
NAMESPACE: supabase
STATUS: deployed
REVISION: 1
NOTES:
---
Thank you for installing knoe-supabase!
==> Installing Supabase Studio/Kong frontdoor release into DB cluster context 'gke_plenary-truck-485623-p7_us-west3_knoe-dev-cnpg-0'...
Release "supabase-frontdoor-db" has been upgraded. Happy Helming!
NAME: supabase-frontdoor-db
LAST DEPLOYED: Mon Apr 13 12:49:32 2026
NAMESPACE: supabase
STATUS: deployed
REVISION: 3
NOTES:
---
Thank you for installing knoe-supabase!
Visit the Studio dashboard at https://api.0.knoe.dev
==> Applying rendered Studio/Kong frontdoor manifests to DB cluster for explicit ingress ownership...
service/supabase-kong configured
ingress.networking.k8s.io/supabase-kong configured
Warning: No nodes found with label 'knoe.dev/node-role=general'; clearing knoe-specific node constraints from Supabase workloads.
Warning: Supabase node selector 'gandalf.prole.org' not found in cluster; clearing hostname selector from Supabase workloads.
==> Waiting for Supabase readiness in namespace 'supabase' (timeout=900s)...
==> Still waiting: elapsed=0s pendingPVCs=0 notReadyPods=6 terminatingPods=0
==> Supabase is Ready (all PVCs bound; all pods ready).
PORT_FORWARD_MAPPING: id=supabase-studio;namespace=supabase;target=svc/studio;address=0.0.0.0;hostPort=8082;servicePort=3000;protocol=TCP;description=Supabase Studio
==> Deployment complete via Helm.
[100%] Supabase deployed
[30%] Deploying Prole...
[100%] Deployment complete
[20%] Rotating master password via update.sh...
[update.sh] === prole update.sh — master password rotation ===
[update.sh] App cluster context : gke_plenary-truck-485623-p7_us-west3_knoe-dev-0
[update.sh] DB cluster context : gke_plenary-truck-485623-p7_us-west3_knoe-dev-cnpg-0
[update.sh] Decrypting master password from vault...
[update.sh] Vault master password decrypted successfully.
[update.sh] Updating DB k8s secrets in knoe-db-0 on DB cluster...
[update.sh] [DONE] DB k8s secrets updated (knoe-db-user, knoe-db-superuser, knoe-db-knoe).
[update.sh] Rotating PostgreSQL user passwords via CNPG primary pod...
[update.sh] Primary pod: knoe-db-1
ALTER ROLE
ALTER ROLE
DO
DO
[update.sh] [DONE] PostgreSQL users rotated (prole, postgres, knoe, authenticator).
[update.sh] Updating Grafana on app cluster (namespace: monitoring)...
[update.sh] [DONE] Grafana k8s secret 'prometheus-grafana' updated.
[update.sh] Grafana pod: prometheus-grafana-0
logger=settings t=2026-04-13T19:50:56.306100701Z level=info msg="Starting Grafana" version=12.4.2 commit=ebade4c739e1aface4ce094934ad85374887a680 branch=release-12.4.2 compiled=2026-04-13T19:50:56Z
logger=settings t=2026-04-13T19:50:56.308009874Z level=info msg="Unified migration configs enforced"
logger=settings t=2026-04-13T19:50:56.30807312Z level=info msg="Enforcing mode 5 for resource in unified storage" resource=playlists.playlist.grafana.app
logger=settings t=2026-04-13T19:50:56.30820073Z level=info msg="Config loaded from" file=/usr/share/grafana/conf/defaults.ini
logger=settings t=2026-04-13T19:50:56.308218692Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_DATA=/var/lib/grafana/"
logger=settings t=2026-04-13T19:50:56.308232403Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_LOGS=/var/log/grafana"
logger=settings t=2026-04-13T19:50:56.308244545Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PLUGINS=/var/lib/grafana/plugins"
logger=settings t=2026-04-13T19:50:56.308257182Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PROVISIONING=/etc/grafana/provisioning"
logger=settings t=2026-04-13T19:50:56.308328435Z level=info msg="Config overridden from Environment variable" var="GF_SECURITY_ADMIN_USER=admin"
logger=settings t=2026-04-13T19:50:56.308344216Z level=info msg="Config overridden from Environment variable" var="GF_SECURITY_ADMIN_PASSWORD=*********"
logger=settings t=2026-04-13T19:50:56.308356814Z level=info msg="Config overridden from Environment variable" var="GF_UNIFIED_STORAGE_INDEX_PATH=/var/lib/grafana-search/bleve"
logger=settings t=2026-04-13T19:50:56.308370746Z level=info msg=Target target=[all]
logger=settings t=2026-04-13T19:50:56.308395353Z level=info msg="Path Home" path=/usr/share/grafana
logger=settings t=2026-04-13T19:50:56.308416244Z level=info msg="Path Data" path=/var/lib/grafana/
logger=settings t=2026-04-13T19:50:56.308426797Z level=info msg="Path Logs" path=/var/log/grafana
logger=settings t=2026-04-13T19:50:56.308448377Z level=info msg="Path Plugins" path=/var/lib/grafana/plugins
logger=settings t=2026-04-13T19:50:56.30865385Z level=info msg="Path Provisioning" path=/etc/grafana/provisioning
logger=settings t=2026-04-13T19:50:56.308676345Z level=info msg="App mode production"
logger=featuremgmt t=2026-04-13T19:50:56.309824064Z level=info msg=FeatureToggles cloudWatchNewLabelParsing=true alertRuleRestore=true awsAsyncQueryCaching=true timeRangePan=true cloudWatchCrossAccountQuerying=true alertingBulkActionsInUI=true grafanaAssistantInProfilesDrilldown=true onlyStoreActionSets=true newTimeRangeZoomShortcuts=true dashgpt=true alertingUIOptimizeReducer=true newLogsPanel=true sharingDashboardImage=true awsDatasourcesTempCredentials=true azureMonitorEnableUserAuth=true alertingUseNewSimplifiedRoutingHashAlgorithm=true dashboardScene=true logsExploreTableVisualisation=true alertingRulePermanentlyDelete=true annotationPermissionUpdate=true alertingRuleRecoverDeleted=true alertingSaveStateCompressed=true improvedExternalSessionHandlingSAML=true lokiQuerySplitting=true newFiltersUI=true lokiLabelNamesQueryApi=true alertingRuleVersionHistoryRestore=true influxdbBackendMigration=true alertingQueryAndExpressionsStepMode=true azureResourcePickerUpdates=true alertingImportYAMLUI=true newPanelPadding=true alertingMigrationUI=true cloudWatchRoundUpEndTime=true grafanaconThemes=true enableSCIM=true improvedExternalSessionHandling=true useSessionStorageForRedirection=true alertingNotificationsStepMode=true kubernetesDashboards=true prometheusAzureOverrideAudience=true logsContextDatasourceUi=true logsPanelControls=true azureMonitorPrometheusExemplars=true publicDashboardsScene=true preventPanelChromeOverflow=true
logger=sqlstore t=2026-04-13T19:50:56.31029072Z level=info msg="Connecting to DB" dbtype=sqlite3
logger=sqlstore t=2026-04-13T19:50:56.311008837Z level=info msg="Using SQLite driver" driver="mattn/go-sqlite3 (CGO enabled)"
logger=sqlstore t=2026-04-13T19:50:56.311421347Z level=warn msg="SQLite database file has broader permissions than it should" path=/var/lib/grafana/grafana.db mode=-rw-rw---- expected=-rw-r-----
logger=migrator t=2026-04-13T19:50:56.32749331Z level=info msg="Locking database"
logger=migrator t=2026-04-13T19:50:56.327792482Z level=info msg="Starting DB migrations"
logger=migrator t=2026-04-13T19:50:56.368334562Z level=info msg="migrations completed" performed=0 skipped=710 duration=2.214834ms
logger=migrator t=2026-04-13T19:50:56.369661454Z level=info msg="Unlocking database"
logger=secrets t=2026-04-13T19:50:56.370719301Z level=info msg="Envelope encryption state" currentprovider=secretKey.v1
Admin password changed successfully ✔
[update.sh] [DONE] Grafana CLI reset successful.
[update.sh] Restarting Grafana StatefulSet to apply new secret...
statefulset.apps/prometheus-grafana restarted
[update.sh] Waiting for Grafana rollout...
Waiting for partitioned roll out to finish: 0 out of 1 new pods have been updated...
Waiting for 1 pods to be ready...
Waiting for 1 pods to be ready...
Waiting for 1 pods to be ready...
partitioned roll out complete: 1 new pods have been updated...
[update.sh] [DONE] Grafana restarted and ready.
[update.sh] Updating password fields in prole.cfg files...
[update.sh] Updated: /home/chrisfu/dev/prole/conf/gke.cfg
[update.sh] Updated: /home/chrisfu/dev/prole/conf/service/prod.cfg
[update.sh] --- Verification ---
[update.sh] [PASS] DB secret knoe-db-0/knoe-db-user
[update.sh] [PASS] DB secret knoe-db-0/knoe-db-superuser
[update.sh] [PASS] DB secret knoe-db-0/knoe-db-knoe
[update.sh] [PASS] Grafana secret monitoring/prometheus-grafana
[update.sh] All secret verifications passed.
[update.sh] === update.sh complete ===
[100%] Security hardening complete
[CONFIG] Wrote conf/gke.cfg
[DONE] Silent install completed.
Reference in New Issue View Git Blame Copy Permalink