fix(net): GCE BackendConfig rejects type: TCP; switch to HTTP /healthz

Follow-up to 391c4f5. Live deploy showed GCE's L7 BackendConfig CRD
hard-rejects type: TCP with:

  Error syncing to GCP: error running backend syncing routine:
  error ensuring health check:
  Protocol "TCP" is not valid, must be one of [HTTP,HTTPS,HTTP2]

The sync never completes, so the LB has no healthy backend and TCP
connections to the public endpoint just close (ERR_CONNECTION_CLOSED).

Fix: switch all three BackendConfigs to type: HTTP with request paths
that return 200:

- supabase-kong & knoe-svc-kong: add a dedicated /healthz route to the
  Kong declarative config via the request-termination plugin, which
  returns 200 synchronously with no upstream call. Equivalent liveness
  semantics to the TCP check we wanted (backend is alive as long as Kong
  accepts connections) but over HTTP, which GCE actually accepts.
  - supabase/helm/knoe-supabase/templates/kong/config.yaml
  - etc/init_kong.sh (inline kong.yml heredoc)

- supabase-studio: Studio returns 301 on / (Next.js default) so we
  point the probe at /favicon.ico -- Next.js serves it as a static asset
  with 200 unconditionally. Not as clean as a real readiness endpoint
  but Studio does not expose one that returns 200 without auth.
  - supabase/helm/knoe-supabase/templates/studio/backendconfig.yaml

Verified locally via helm template -f values.generated.json: the
rendered BackendConfigs come out with the HTTP protocol + correct paths,
and the Kong ConfigMap has the healthz service block before the
auth-v1-open service.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

etc/init_kong.sh

@@ -378,6 +378,23 @@ _format_version: "3.0"
 _transform: true
 
 services:
+  # Dedicated health endpoint for the GCE LB BackendConfig. request-termination
+  # returns 200 synchronously with no upstream call, so the probe passes as long
+  # as the Kong proxy itself is running. GCE rejects \`type: TCP\` in BackendConfig
+  # (only HTTP/HTTPS/HTTP2 accepted), so we use this route for HTTP liveness.
+  - name: healthz
+    url: http://127.0.0.1:${KONG_PROXY_PORT}/
+    routes:
+      - name: healthz
+        paths:
+          - /healthz
+        strip_path: true
+    plugins:
+      - name: request-termination
+        config:
+          status_code: 200
+          message: ok
+
   - name: prole-service
     url: ${PROLE_SERVICE_UPSTREAM_URL}
     routes:
@@ -608,9 +625,13 @@ EOF
 
     # BackendConfig: GCE default healthCheck is HTTP GET / on the backend port
     # and Kong returns 404 on an unrouted path, so the backend never goes
-    # HEALTHY. A TCP healthCheck on the proxy port is sufficient for our case
-    # (backend is alive as long as Kong is accepting connections). Service is
-    # annotated below with cloud.google.com/backend-config so GCE picks this up.
+    # HEALTHY. We wanted TCP (Kong is alive as long as it accepts connections),
+    # but GCE's BackendConfig CRD rejects `type: TCP` with
+    #   `Protocol "TCP" is not valid, must be one of [HTTP,HTTPS,HTTP2]`
+    # so we fall back to HTTP against the `/healthz` route we add to the
+    # knoe-svc-kong declarative config above (request-termination plugin
+    # returns 200 synchronously, no upstream dependency -- equivalent liveness
+    # semantics to a TCP check but over a protocol GCE accepts).
     echo "Reconciling BackendConfig (${SVC_KNOE_BACKEND_CONFIG_NAME}) for ${KONG_NAME} in ns=${NAMESPACE} ..."
     kubectl apply -f - <<EOF
 apiVersion: cloud.google.com/v1
@@ -620,7 +641,8 @@ metadata:
   namespace: ${NAMESPACE}
 spec:
   healthCheck:
-    type: TCP
+    type: HTTP
+    requestPath: /healthz
     port: ${KONG_PROXY_PORT}
     checkIntervalSec: 15
     timeoutSec: 5

supabase/helm/knoe-supabase/templates/kong/backendconfig.yaml

@@ -3,10 +3,17 @@ GCE L7 BackendConfig for supabase-kong.
 
 GCE's default Ingress healthCheck is HTTP GET `/` on the backend port; Kong
 returns 404 on any unrouted path, so the backend never goes HEALTHY and the
-LB serves "Server Error" instead of reaching the proxy. A TCP healthCheck on
-the Kong proxy port is sufficient for our deployment shape -- the backend is
-"alive" as long as Kong is accepting connections, and GCE only probes for
-reachability; per-route health is not required here.
+LB serves "Server Error" instead of reaching the proxy.
+
+We wanted TCP liveness semantics (backend is alive as long as Kong accepts
+connections) but GCE's BackendConfig CRD only accepts HTTP/HTTPS/HTTP2 --
+it rejects `type: TCP` with
+  `Protocol "TCP" is not valid, must be one of [HTTP,HTTPS,HTTP2]`
+
+Workaround: HTTP against a dedicated `/healthz` route we add to Kong's
+declarative config (templates/kong/config.yaml) with the request-termination
+plugin returning 200 synchronously, no upstream call. Equivalent liveness
+semantics via a protocol GCE accepts.
 
 The companion Service template (kong/service.yaml) annotates the Service
 with cloud.google.com/backend-config so GCE picks this up.
@@ -23,7 +30,8 @@ metadata:
     {{- include "supabase.labels" . | nindent 4 }}
 spec:
   healthCheck:
-    type: TCP
+    type: HTTP
+    requestPath: /healthz
     port: {{ .Values.service.kong.port | default 8000 }}
     checkIntervalSec: 15
     timeoutSec: 5

supabase/helm/knoe-supabase/templates/kong/config.yaml

@@ -48,6 +48,25 @@ data:
         password: ${DASHBOARD_PASSWORD}
     {{- end }}
     services:
+      # Dedicated health endpoint used by the GCE LB BackendConfig.
+      # request-termination returns 200 synchronously without hitting any
+      # upstream, so the probe passes as long as the Kong proxy itself is
+      # accepting requests -- same liveness semantics as the TCP check we
+      # originally wanted, but using the HTTP protocol that GCE's L7
+      # BackendConfig CRD actually accepts (TCP is rejected with
+      # `Protocol "TCP" is not valid, must be one of [HTTP,HTTPS,HTTP2]`).
+      - name: healthz
+        url: http://127.0.0.1:8000/
+        routes:
+          - name: healthz
+            strip_path: true
+            paths:
+              - /healthz
+        plugins:
+          - name: request-termination
+            config:
+              status_code: 200
+              message: ok
     {{- if .Values.deployment.auth.enabled }}
       - name: auth-v1-open
         url: http://{{ include "supabase.auth.fullname" . }}:{{ .Values.service.auth.port }}/verify

supabase/helm/knoe-supabase/templates/studio/backendconfig.yaml

@@ -2,9 +2,15 @@
 GCE L7 BackendConfig for supabase-studio.
 
 Mirror of kong/backendconfig.yaml. Studio is a Next.js app on port 3000
-that returns 301 redirects on `/` (not 200), which trips GCE's default
-HTTP healthCheck and marks the backend UNHEALTHY. A TCP healthCheck is
-sufficient for LB-level liveness.
+that returns 301 redirects on `/` (not 200), tripping GCE's default HTTP
+healthCheck and marking the backend UNHEALTHY.
+
+We wanted TCP liveness semantics but GCE's BackendConfig rejects `type:
+TCP` (see kong/backendconfig.yaml for the exact error). Workaround: hit
+`/favicon.ico` -- Next.js serves the favicon as a static asset and
+returns 200 unconditionally, so the probe passes as long as the Next.js
+server is up. Not as clean as a real readiness endpoint, but Studio does
+not expose one that returns 200 without auth.
 
 Rendered only when .Values.service.studio.backendConfigName is set.
 */ -}}
@@ -17,7 +23,8 @@ metadata:
     {{- include "supabase.labels" . | nindent 4 }}
 spec:
   healthCheck:
-    type: TCP
+    type: HTTP
+    requestPath: /favicon.ico
     port: {{ .Values.service.studio.port | default 3000 }}
     checkIntervalSec: 15
     timeoutSec: 5