Commit Graph

7 Commits

Author SHA1 Message Date
chrisfu
a5cd0c45f4 Update conf, k8s registry, knoe UI screens, etc init scripts, and scan data
- conf/service/prole.cfg, port-mapping.cfg, cnpg-placement updated

- k8s/registry/deployment.yaml updated

- knoe/ui/screens: base.py, knoe_users.py, __init__.py updated

- etc/ init scripts refreshed (gitlab, knoe_users, registry, prole_cfg)

- modes/k3s/knoe-db/.version bumped; scan network_description and ansible_inventory updated

Co-authored-by: Junie <junie@jetbrains.com>
2026-04-02 23:40:16 -07:00
chrisfu
15890187a3 Stabilize CNPG reset/update flow and finalize 3-node recovery
- add idempotent CNPG operator reconciliation to prevent duplicate active controller ReplicaSets
- restore and validate CNPG 3-instance convergence with PV claimRef recovery and storage path readiness
- wire canonical launcher aliases/reset behavior and improve namespace cleanup semantics
- harden backup/objectstore readiness handling and retry behavior
- enforce service namespace usage for common services and remove default-namespace drift
- enable Kerberos milestone auto-activation when realm/kdc are configured and apply CNPG GSS pg_hba rules
- keep final runtime healthy: knoe-db 3/3, operator stable, perfsnap captured

Co-authored-by: Junie <junie@jetbrains.com>
2026-03-27 21:54:53 -07:00
chrisfu
65d03ed442 stable: slimmed Supabase + CNPG 3-node healthy build (2026-03-25)
STABLE BUILD — all 35+ pods Running, zero non-Running pods, CNPG ready=3.

## Architecture changes (Supabase slim-down)
- Removed local Supabase Kong, Storage, MinIO, Imgproxy from deployed stack
- Rewired API routing to shared Kong in kube-system
- Rewired object storage to shared Garage in knoe-system
- Supabase footprint reduced to control-plane only: auth, rest, meta,
  analytics, realtime (studio port-forward retained)
- Supabase pods renamed: stripped redundant '-prole-supabase-' prefix via
  per-component fullnameOverride (e.g. supabase-auth, supabase-rest, ...)
- Supabase workloads pinned to merlin.prole.org (memory-safe node)

## File changes
- conf/service/prole.cfg
    * Replaced all hardcoded /Users/chrisfu/dev/prole paths with ${HOME}/dev/prole
      so config is portable when run from myrddin.prole.org or any user home
    * SUPABASE_ENABLED=True, SUPABASE_PRIMARY_NODE=merlin.prole.org in
      both [Global] and [Optional Features]
    * supabase_enabled=true in [Inputs]/init_cluster section
- supabase/helm/render_supabase.py
    * Explicitly disabled: kong, storage, minio, imgproxy
    * Active components pinned via nodeSelector to merlin.prole.org
    * fullnameOverride per component to clean up pod names
    * Default primary node changed from pi.prole.org -> merlin.prole.org
- supabase/deploy.sh
    * Removed svc/kong port-forward; API via shared Kong in kube-system
    * Removed imgproxy/storage from PV/PVC prep and helm --set persistence args
    * Fixed kubectl exec missing -i flag for psql heredoc stdin in
      setup_knoe_db_for_supabase
    * Updated Studio access messaging to reference shared platform Kong
- k8s/prole/knoe-db.yaml
    * instances: 3
    * Added control-plane/master tolerations for myrddin scheduling
- deploy/opentofu/k3s/manifests/prole/knoe-db.yaml
    * instances: 3, added control-plane tolerations (mirrors k8s/prole/)
- k8s/prole/iscsi-pvs.yaml
    * Expanded d003-data/d003-wal nodeAffinity to [pi, merlin, myrddin]
      to reflect post-OOM-recovery live state
- k8s/registry/deployment.yaml
    * nodeSelector restored to myrddin.prole.org (hostPort 5000 owner)
    * Added control-plane + master tolerations so pod schedules correctly
- modes/k3s/knoe-db/.version
    * Aligned to 140 (image tag 18-140) matching repo-root version
- etc/init_common_services.sh, etc/init_registry.sh
    * Minor fixes aligned with slimmed service layout and node targeting
- infrastructure/inventory/host_vars/pi.prole.org.yml
    * Updated host vars to reflect post-recovery pi state
- knoe/core/topology.py, knoe/ui/screens/cluster.py,
  knoe/ui/screens/cluster_nodes.py
    * Topology and UI updates for 3-node cluster representation
- scan/network_description.txt
    * Refreshed network scan output post-recovery
- conf/port-mapping.cfg
    * Updated port mapping to reflect removal of local Kong/storage forwards
- scripts/cleanup_stale_rs.sh (new)
    * Utility script to prune orphaned ReplicaSets after rollouts

## Verified stable state
- CNPG: Cluster in healthy state, ready=3
    knoe-db-2 (merlin, primary), knoe-db-3 (pi), knoe-db-5 (myrddin)
- Supabase: 5 pods Running on merlin.prole.org
    supabase-auth, supabase-rest, supabase-meta,
    supabase-analytics, supabase-realtime
- Monitoring: all prometheus/grafana/alertmanager pods Running on merlin
- Registry: Running on myrddin.prole.org (hostPort 5000)
- Zero non-Running / non-Completed pods across all namespaces

Co-authored-by: Junie <junie@jetbrains.com>
2026-03-25 11:56:40 -07:00
chrisfu
598eae4633 Enable TLS for k3s registry and deploy SSL certs
- Add ssl_hosts group and prole_ssl role to deploy registry cert/key to /etc/ssl/certs/prole

- Configure k8s registry deployment to serve HTTPS using mounted host certs

- Switch k3s/containerd registry mirror config to https:// and prefer HTTPS checks with HTTP fallback

Co-authored-by: Junie <junie@jetbrains.com>
2026-03-10 00:58:08 -07:00
chrisfu
ec2795f8a3 Checkpoint: Kerberos egress fixes, local registry provisioning for k3d/k3s, and installer GUI workflow enhancements. Key changes: AD forwarder port offsets, outbound NetworkPolicy, KDC dns_lookup disabling; k3d/k3s registry support with Docker network connect fix; GUI 'Deploy Services' button and manual pause; new test_service_layer_navigation.py. 2026-02-17 14:26:12 -08:00
chrisfu
96f594fd3c Refactor initialization scripts and add new service components
- Consolidated and split initialization scripts in etc/:
    - Removed init_prole-db.sh and init_authority.sh.
    - Added init_kdc.sh for in-cluster MIT Kerberos KDC (prole-authority).
    - Added init_ollama.sh for Ollama AI service integration.
    - Added init_service_layer.sh for high-level service orchestration.
    - Added init_k3s_registry.sh for private registry management.
- Major updates to install.py:
    - Support for new Ollama and KDC configuration.
    - Improved prole.cfg rendering and namespace handling.
    - Updated unattended install flags.
- Infrastructure and Deployment:
    - Updated K3s Ansible role with private registry support (registries.yaml template).
    - Added prole-authority Dockerfile.
    - Updated OpenBao Kerberos ConfigMap and other K8s manifests.
- Configuration:
    - Updated prole.cfg with new sections for Ollama and Monitoring.
    - Refined environment variable exports in env.sh and prole_cfg.sh.
2026-02-11 13:09:31 -08:00
chrisfu
87e2f5d385 Checkpoint: Refactor common services and monitoring initialization. Updates: etc/init_monitoring.sh (Grafana naming, legacy cleanup), etc/init_common_services.sh (migration/cleanup logic), added etc/init_registry.sh, updated etc/init_cloudnative_pg.sh (manifest filtering), updated configs, and enhanced tests. Fixed: excluded .ansible logs and added to .gitignore. 2026-02-09 18:50:44 -08:00