vault_samba_dns_admin_pass was stale — samba-tool commands were failing
with NT_STATUS_LOGON_FAILURE. Updated to match current AD Administrator
credential so the samba_dns role can manage records end-to-end.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
All samba-tool dns commands were using -P (machine account Kerberos) which
silently failed with no error — tasks reported changed=0 but records were
never written. Switch to -U Administrator --password={{ samba_dns_admin_pass }}
which uses the vault-protected admin credentials that were already defined
but never wired up.
Also fix regex patterns in record parsers: samba-tool output uses `A: IP`
and `PTR: fqdn` format, not `A IP` / `PTR fqdn` (space-separated), so
updated regex_findall patterns to match `TYPE:\s+value`.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- knoe-db.yaml: switch to CNPG-managed TLS cert with serverAltDNSNames
(pg.prole.org + knoe-db-rw cluster service) — removes static serverTLSSecret/serverCASecret
- dns.yml: add pg.prole.org A record to prole_k3s_dns_records (10.0.0.3, 10.0.0.6)
for Ansible-managed split-horizon DNS via Samba AD DC
- k3s.cfg: align KNOE_HOME paths to ~/dev/prole, add PROLE_KDC_* vars, remove
hardcoded KUBECTL_CONTEXT (kubeconfig current-context is authoritative)
- prod.cfg: add PROLE_KDC_STORAGE_CLASS = prole-iscsi
- onepassword.py: skip vault check gracefully when no 1Password session active (non-TTY)
- CLAUDE.md: document production postgres connection string and DNS/CA cert ops
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The matching knoe-db change (commit `ff7546d`) re-keys the MIT-side
cross-realm krbtgts with `-e arcfour-hmac:normal` to dodge the
AES-salt mismatch between Samba and MIT. For the trust to actually
carry traffic, the Samba side must also offer only RC4 on the
trust account — otherwise the client picks AES, the two sides
derive different keys from the same password (different salt
conventions), and TGT decryption fails.
Change `msDS-SupportedEncryptionTypes` from 28 (RC4+AES128+AES256)
to 4 (RC4 only). RC4 has no salt, so keys derive from the password
alone and both sides converge.
Updated docstring + summary debug print to reflect the new value
and the why.
Tracked alongside the Junie brief at
~/dev/knoe-db/docs/plans/junie/kdc-trust-reset-repeatable.md.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
klist shows the Kerberos principal (slash form
krbtgt/KNOE.LOCAL@PROLE.ORG), not the sAMAccountName underscore form
(krbtgt_KNOE.LOCAL). Update the summary debug task accordingly and
split the kvno hint into two stages (the cross-realm TGT first, then
a service ticket) so the user can isolate which step fails when
chasing salt/enctype issues.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The apt install task was positioned after the user-create step, so a
failure in create (or anywhere upstream) meant ldb-tools never got
installed. Move the apt task to the very top of the play, before the
cluster-secret lookups, so it always runs first on every invocation.
Drop the duplicate apt task that previously sat just before the
ldbmodify locator.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The _trust_exists probe was still grepping samba-tool user list for
"krbtgt/KNOE.LOCAL" (slash form), but the actual sAMAccountName is
"krbtgt_KNOE.LOCAL" (underscore form, since slash is invalid in
sAMAccountName). _trust_exists was therefore always false, the create
step retried against the existing account on every re-run, and Samba
rejected it as a name collision.
Match against samba_account_name directly so re-runs route through
the setpassword branch instead.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
myrddin doesn't have ldbmodify installed — it's in the ldb-tools
Debian package, separate from samba-tool. Add an apt task ahead of
the locator so the package is in place before we try to use it.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
ldbmodify is part of the ldb-tools Debian package but is not in
root's default PATH under sudo on the Samba DC, so the user-attribute
modify step failed with "ldbmodify: command not found".
Add a locator task that probes the conventional install paths
(/usr/bin, /usr/sbin, /usr/local/bin, /opt/samba/bin) and falls back
to a depth-limited find under /usr and /opt. The downstream
ldbmodify invocation now uses the absolute path resolved here.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
samba-tool user create rejected the cluster-generated trust_shared_password
with "0000052D: Constraint violation - check_password_restrictions: the
password does not meet the complexity criteria". The trust password is
high-entropy random bytes from init_kdc.sh and may not satisfy AD's
"3 of 4 character categories" rule.
Wrap the create + setpassword steps in a block that:
1. reads the current `samba-tool domain passwordsettings show` state
2. flips complexity off if it was on
3. does the user write
4. always: restores complexity to its prior on/off state
The restore is in an `always:` clause so a failure inside the write
block does not leave the domain policy weakened.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
samba-tool user create rejected the sAMAccountName "krbtgt/KNOE.LOCAL"
with "samldb: sAMAccountName contains invalid '/' character". AD/Samba
disallow "/" in sAMAccountName even though Kerberos principal names
require it (krbtgt/REMOTE@LOCAL).
Switch storage name to "krbtgt_KNOE.LOCAL" and add an ldbmodify pass
that sets the canonical Kerberos identity on the same object:
userPrincipalName = krbtgt/KNOE.LOCAL@PROLE.ORG
servicePrincipalName = krbtgt/KNOE.LOCAL
msDS-SupportedEncryptionTypes = 28 (RC4+AES128+AES256)
Samba KDC resolves principals by UPN/SPN, so a TGS-REQ for
krbtgt/KNOE.LOCAL@PROLE.ORG will hit this account.
Note: key-salt parity with the MIT side is NOT guaranteed yet. Samba's
default salt for AES keys is REALM+UPN; MIT's default for cross-realm
krbtgt is REALM+"krbtgt"+REMOTE. If kvno fails with "decrypt integrity
check failed" we'll add a keytab-export/import step in a follow-up
rather than try to coerce Samba's salt at creation time.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
In --check mode the user-create step is skipped (it's mutating), so
the subsequent `samba-tool user show krbtgt/KNOE.LOCAL` probe fails
with rc=1 on a first dry-run because the user doesn't yet exist. Add
the same guarded failed_when used on the trust validate/show tasks
earlier: only fail when NOT (check_mode AND trust didn't pre-exist).
Real runs still fail hard on rc!=0.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
samba-tool domain trust create --type=external only works against
another writeable AD-style DC; against an MIT KDC it fails with
"Failed to find a writeable DC for domain 'KNOE.LOCAL'". Our peer at
svc/auth in knoe-system is a plain Kerberos KDC, so we cannot use that
path.
Replace the trust create/validate/show steps with the supported
Samba-AD <-> MIT-KDC approach: create the inter-realm TGT principal as
an ordinary user account in Samba whose sAMAccountName is
"krbtgt/KNOE.LOCAL", with its password synced to the cluster Secret
knoe-system/knoe-kdc-secrets/trust_shared_password (which the MIT side
already keys against). Steps:
1. samba-tool user list (idempotency probe)
2. samba-tool user create krbtgt/KNOE.LOCAL ... (when missing)
3. samba-tool user setpassword krbtgt/KNOE.LOCAL (when present, to
re-sync after a cluster Secret rotation)
4. samba-tool user setexpiry --noexpiry
5. ldbmodify msDS-SupportedEncryptionTypes=28 (RC4+AES128+AES256
to match what init_kdc.sh sets on the MIT side)
6. samba-tool user show (smoke probe)
7. debug task prints the manual kvno smoke-test command
Cluster-secret lookups and krb5.conf [realms]/[domain_realm] edits are
unchanged - those parts were correct.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Ansible --check mode auto-skips command/shell tasks unless they
explicitly opt in via check_mode:false. The two cluster-state lookups
(svc/auth ClusterIP and knoe-kdc-secrets/trust_shared_password) were
silently skipped during dry-run, leaving trust_kdc_ip empty and tripping
the assertion immediately.
Add check_mode:false to the five read-only tasks so a --check run can
still resolve cluster state and report what would change:
- Resolve trust_kdc_ip from cluster
- Resolve trust_shared_password from knoe-kdc-secrets
- samba-tool domain trust list (idempotency probe)
- samba-tool domain trust validate
- samba-tool domain trust show (smoke print)
The mutating create step (samba-tool domain trust create) keeps the
default behavior - skipped in check mode. validate/show gain a guarded
failed_when so a check-mode dry-run on a host where the trust does not
yet exist does not fail (the create was skipped, so a non-zero rc is
expected). Real runs still fail hard on rc!=0.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The cross-realm trust playbook was written for the pre-rebrand realm name
PROLE.LOCAL, which no longer exists in the in-cluster KDC — the canonical
realm is now KNOE.LOCAL. The Samba AD trust was therefore never actually
established, leaving chrisfu@KNOE.LOCAL→chrisfu@PROLE.ORG service ticket
flows blocked (and blocking pg_oauth / db.prole.org Kerberos work).
Changes:
- Realm: PROLE.LOCAL → KNOE.LOCAL across all task names, vars, and the
krb5.conf [realms] / [domain_realm] blocks added on myrddin.
- samba_admin_password now resolves from the Ansible vault var
vault_samba_dns_admin_pass (group_vars/ad_dc/vault.yml) by default,
with SAMBA_ADMIN_PASSWORD env and -e overrides preserved for CI.
- trust_shared_password auto-resolves from the in-cluster Secret
knoe-system/knoe-kdc-secrets (key trust_shared_password) when not
passed explicitly — same Secret init_kdc.sh writes.
- Added [domain_realm] mapping for *.knoe.local → KNOE.LOCAL so Samba
can resolve in-cluster service principals.
- Added a final `samba-tool domain trust show` smoke step so a successful
run prints the visible trust state for log review.
Run:
ANSIBLE_VAULT_PASSWORD_FILE=$PWD/.vault_pass ansible-playbook infrastructure/playbooks/kerberos_trust_setup.yml
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
win_regedit writes HKLM keys without privilege escalation when
ansible_user is already a local/domain Administrator. become:true
with --ask-become-pass was overriding the playbook's runas method.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
A single play with become:true uses sudo by default, which Ansible's
Windows exec wrapper rejects immediately at gather_facts. Split into:
- Play 1: hosts workstations,!workstations_windows — macOS/Linux, sudo
- Play 2: hosts workstations_windows — Windows, become_method:runas
Also set morgana to ansible_connection=local (running Ansible from
morgana itself, so SSH self-lookup fails DNS resolution).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add morgana and zinfandel (macOS) and morgoth + fairyland (Windows)
to the workstations inventory.
Extend workstation_kerberos.yml to handle Windows targets:
- Chrome and Edge AuthServerAllowlist set via win_regedit under
HKLM\SOFTWARE\Policies\{Google\Chrome,Microsoft\Edge}
- krb5.conf and plist tasks skip on Windows (uses built-in SSPI/AD)
- Windows hosts in [workstations_windows] group with WinRM/NTLM
transport; switch to kerberos transport after domain join
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Problem: 'defaults write' is not an onboarding path, it's a support
incident waiting to happen. Browser SPNEGO must be deployed via managed
policy at machine provisioning time, not per-user.
Changes:
etc/init_knoe_users.sh
- _op_ensure_auth(): tries OP_SERVICE_ACCOUNT_TOKEN env, k8s secret
'op-service-account-token', existing interactive session, and op-run
inheritance — in that order. Gives actionable instructions when none
work, including the correct 'op run --' invocation documented in k3s.cfg.
- gitea_ensure_password(): calls _op_ensure_auth() at entry; eliminates
the silent 'op not authenticated' failure path.
- Script header: documents 'op run -- bash etc/init_knoe_users.sh' as
the intended invocation for admin's laptop.
- 'Next steps' output: replaces 'defaults write' with reference to
workstation_kerberos.yml Ansible playbook.
infrastructure/playbooks/workstation_kerberos.yml (new)
- Deploys /etc/krb5.conf (PROLE.ORG realm, myrddin KDC) to all managed
endpoints.
- Deploys Chrome + Edge managed policy (AuthServerAllowlist) on macOS
and Linux — no per-user browser configuration ever required.
- Idempotent; run during laptop provisioning or re-run at any time.
- Targets 'workstations' Ansible group.
infrastructure/inventory/hosts.ini
- Adds [workstations] group with example entries and onboarding notes.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
selectattr with 'match' regex was returning empty sequence on this Ansible
version. Since each install entry has exactly two items (fullchain.cer +
key), rejectattr the fullchain to get the key — no regex needed.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The dns_namecom.sh plugin uses Namecom_Username and Namecom_Token (not
NAMECOM_*). Fix the environment dict, the SAVED_ keys written to
account.conf, and add a cleanup task to remove the stale all-caps entries
already present on hosts from previous runs.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Re-adds no_log: true removed in 5e4e17e and switches the issue task from
ansible.builtin.shell with inline VAR=value to ansible.builtin.command with
an environment: dict. This prevents credentials appearing in Ansible failure
output and avoids the inline env var being clobbered when acme.sh sources
account.conf internally before the dns_namecom plugin runs.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Ansible command module + become does not reliably pass environment:
vars through sudo env_reset. Use ansible.builtin.shell with POSIX
inline assignment (VAR=val cmd) so credentials are set inside the
sudo-spawned shell subprocess, never touching the sudo env barrier.
account.conf writing retained for cron renewal.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
_readaccountconf_mutable checks SAVED_NAMECOM_Username first; if that
key lookup exits 0 with empty output it short-circuits and never reads
the bare NAMECOM_Username key. Write the SAVED_ prefix that acme.sh
uses internally when persisting credentials.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
sudo env_reset strips the environment: key through become. Write
NAMECOM_Username and NAMECOM_Token to acme.sh's account.conf directly,
which dns_namecom reads as a fallback via _readaccountconf_mutable.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- New acme role: installs acme.sh, issues certs for myrddin/merlin/gandalf
via DNS-01 (name.com API), installs cert files to /etc/ssl/certs/prole/,
sets up renewal cron. rc=2 (already valid) treated as ok.
- New playbooks/acme.yml: runs acme role on myrddin (issues all 3 host certs)
- New playbooks/certmgr.yml: cert-manager and letsencrypt-prod ClusterIssuer
(HTTP-01/Traefik) for external endpoints (db/svc/api.prole.org)
- Add vault references for name.com API creds to group_vars/all/vars.yml
- Add prole_ssl_src_dir and prole_ssl_files to myrddin/merlin/gandalf host_vars
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add gandalf.prole.org → 10.0.0.7 to prole_dns_records and
prole_internal_a_records in dns.yml (was missing, causing nslookup
to fail on myrddin)
- prole_ssl: stat-check each source file on the controller before
copying; emit a warning and skip rather than hard-failing when certs
have not yet been placed in ssl/prole/
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Installs Tailscale on k3s agents via the Debian stable apt repo using
the modern signed-by keyring approach. Auth key stored in vault.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The ansible excision in 5a769f2 was correct in scope -- no more host
inventory, no playbooks, no roles -- except it took out the one file
update.sh depends on for master DB password rotation:
infrastructure/inventory/group_vars/all/vault_db_master.yml
update.sh(1) reads this file (ansible-vault encrypted) at the start of
every run to decrypt the master password used to rotate k8s DB secrets
(knoe-db-user/superuser/knoe), ALTER ROLE on CNPG primary for the
prole/postgres/knoe/authenticator users, and reset Grafana admin creds.
With the vault file gone, update.sh failed with
ERROR: Vault file not found: .../vault_db_master.yml
and took down the tail of deploy.sh.
Restored from the parent commit of 5a769f2 verbatim -- content is
already ansible-vault AES256 encrypted, so it's safe to carry in a
private repo. The `.vault_pass` file that decrypts it remains
gitignored (never committed). Broader `infrastructure/` stays gone.
TODO (deferred): migrate this one remaining ansible-vault secret to
either OpenBao or a k8s Secret so knoe-db doesn't retain a residual
ansible-vault dependency surface. Tracked in our pending work.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
knoe-db deploys cluster-native (mode k3d for local dev, k8s for the
shared company cluster, k3s still supported for individual homelab
collaborators via conf/k3s.cfg). The ansible-managed distributed cluster
of prole.org machines is no longer the target topology, and the Ansible
inventory in this tree targets only the original author's specific hosts
(merlin/gandalf/myrddin/pi/raspberry/retropie.prole.org). It is neither
portable nor useful for the new deployment shape.
Removed:
- infrastructure/inventory/ (hosts.ini + 7 *.prole.org host_vars)
- infrastructure/playbooks/ (11 playbooks, all hardcoded to *.prole.org)
- infrastructure/roles/ (20 roles: pihole, samba AD DC, mariadb,
netplan, iscsi, prole_ssl, etc. -- all
tied to the bare-metal prole.org topology)
- infrastructure/setup/ (cr_ansible_user.sh, cr_samba_family_users.sh,
filesystem.txt -- prole.org bootstrap)
- infrastructure/deployments/svc-check-helm/
(legacy; etc/init_kong.sh already tears
down svc-check resources on every run)
- infrastructure/ansible.cfg (references deleted inventory)
- ansible.cfg, ansible.sh, ansible_min.cfg, ansible_recovery.log (root;
orphaned without inventory)
- vault_backup/ (ansible-vault encrypted prole.org secrets
backup: ad_dc_vault.yml, all_prole_vault.yml,
iscsi_vault.yml)
- tools/ (dashboard.sh, ghcr_docker_login.sh,
k3s-cluster-rename.sh, manage-node.sh,
export_mariadb_synology.sh,
prepare_mariadb_usb.sh, and a dumped
synology.prole.org mariadb .sql file --
all prole.org-era deployment helpers)
Kept: scripts/ (cluster ops), bin/ (prole-env.sh, prole-kpf.sh dev
shims), conf/k3s.cfg / conf/k3d.cfg (still supported modes; any
*.prole.org defaults in those files are legitimate mode fallbacks for
collaborators who keep running their own k3s/k3d setups). Test
fixtures referencing *.prole.org hostnames are left as-is (regression
data).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Summarize CLAUDE.md updates:
- Document both live GKE clusters as e2-standard-2 x3 in us-west3 and note stale app default in code.
- Call out DB cluster naming mismatch between conf (knoe-dev-cnpg-0) and code default (knoe-cnpg-0).
- Add SSD quota guidance: 300GB fully consumed by CNPG, so non-CNPG PVCs must use standard/pd-standard.
- Refresh reset script reference and quota section details to match current runtime state.
Co-authored-by: Junie <junie@jetbrains.com>
Rename env config files from conf/*/prole.cfg to conf/k3d.cfg, conf/k3s.cfg, and conf/gke.cfg. Update shell/Python loaders and etc/deploy scripts to resolve named configs cleanly while keeping legacy fallback behavior. Align k3s Ansible tasks, docs, and regression coverage with the new configuration layout.
Co-authored-by: Junie <junie@jetbrains.com>
- fix conf/prod/prole.cfg by replacing hardcoded /Users/chrisfu paths with /Users/chrisfu
- remove injected [update.sh] log lines and stray password artifacts so config is executable
- include latest pending updates across deployment config, UI/core flow, vault/network artifacts, and helper scripts
## GCP / Cluster Environment Screen
- Auto-populate Cloud tab from conf/prod/gcp.cfg on screen open (org_id,
billing_account, billing_project, project_id)
- gcloud auth validity checked on screen startup; friendly modal dialog
streams gcloud auth login output live so user never leaves the app
- Live GKE cluster browser: fetches clusters via gcloud container clusters
list, displays with checkmark selector, auto-selects saved cluster
- Selecting a cluster runs get-credentials, sets KUBECONFIG/KUBECONTEXT,
and syncs the region dropdown to the selected cluster's location
- Region dropdown populated live from gcloud compute regions list with
checkmark on currently selected region; graceful fallback when offline
- New 'GCP Storage' tab with workload->StorageClass mapping (CNPG->premium-rwo,
Redis/Monitoring->standard-rwo, Garage->garage-hdd) and Fetch from Cluster
- Provider readonly field styled correctly (no solid-black on macOS)
- Stale prole.cfg/conf/prole.cfg symlinks removed; all config I/O now
resolves env-specific paths via prole_conf.entrypoint_path()
## GKE Autopilot Compatibility (Common Services)
- Synology iSCSI StorageClass and static PVs guarded behind PROLE_MODE!=k8s
in init_openbao.sh (GKE Autopilot forbids hostPath/iSCSI volumes)
- In-cluster Docker registry (hostPath) skipped in k8s mode; GCP Artifact
Registry used instead
- Kong renamed knoe-svc-kong in k8s mode; all health-check kubectl calls in
init_common_services.sh and status_common_services.sh updated accordingly
- DNS endpoints switched from *.prole.org to *.knoe.dev in k8s mode
(api.knoe.dev, git.knoe.dev, svc.knoe.dev); ingress uses gce class
- New GKE-clean Kong manifests under deploy/opentofu/k8s/manifests/prole/:
no k3s node affinity, explicit Autopilot resource requests/limits
## Garage S3 Store (GKE)
- New garage-statefulset-gcp.yaml targeting garage-hdd StorageClass
(pd-standard, avoids SSD_TOTAL_GB quota exhaustion in us-west3)
- New storageclass-gcp-hdd.yaml (pd-standard, Retain, WaitForFirstConsumer)
- GCP StorageClass manifests skipped on re-runs (Autopilot built-ins are
immutable; skip-if-exists guard added)
- PVC deletion guard extended to cover any storageClass (not just synology)
so stale claims are cleaned before StatefulSet recreation
## Topology (GKE Autopilot)
- DaemonSet collector skipped in prod mode (forbidden in kube-system by
GKE Warden); Kubernetes-only node facts path used instead
- All ready GKE nodes assumed cnpg-eligible and monitoring-eligible without
taint/synology-mount checks (skip_collector + assume_nodes_eligible flags)
## KUBECONFIG / kubectl (k8s mode)
- actions.py: new elif mode==k8s branch sets KUBECONFIG=~/.kube/config
and injects KUBECONTEXT from prole_cfg_data into script env
- _build_kubectl_cmd falls back to Global.KUBECONTEXT when
init_cluster.selected_kubectx is empty
- _activate_selected_gke_cluster persists KUBECONFIG/KUBECONTEXT to
prole_cfg_data and saves prole.cfg immediately after get-credentials
## Database Build Screen (GKE)
- Registry display shows correct Artifact Registry URL
(<region>-docker.pkg.dev/<project>/<namespace>/knoe-db) in green
- Build+push: gcloud auth configure-docker, auto-creates AR repository
named after SERVICE_NAMESPACE (e.g. knoe-system) if missing, then
docker tag + push; falls back to gcr.io if region unavailable
- GCP config loaded from conf/prod/gcp.cfg on every screen entry;
keys normalised to lowercase so project_id lookup is always consistent
## Config / Namespace persistence
- prole_conf.py activate_environment: symlink creation removed; sets
CLUSTER_ENV env-var so all subsequent calls resolve correct env directory
- knoe/ui/screens/__init__.py: startup config load uses entrypoint_path()
instead of hardcoded conf/prole.cfg; seeds SERVICE_NAMESPACE=knoe-system
for managed envs so Common Services never defaults to 'default'
- cfg.py _save_prole_cfg: saves to env-specific path via entrypoint_path()
- etc/prole_cfg.sh: removed all ln -snf symlink creation
Co-authored-by: Junie <junie@jetbrains.com>
- add tools/manage-node.sh for k3s node add/remove automation with inventory reconciliation
- include kubectl drain flag compatibility and --ssh-host targeting for uninstall workflows
- persist installer cfg/env values and update deploy/database/cfg UI handling
- refresh related inventory/network artifacts and expand installer/database test coverage
Co-authored-by: Junie <junie@jetbrains.com>
Update conf/service/prole.cfg to persist the k3s kube context as knoe.dev.prole.org and replace host-specific absolute paths with $HOME-based paths for portability.
Include all pending project changes: local_user role package and screenrc provisioning updates, plus tools/k3s-cluster-rename.sh and tests/etc/test_k3s_cluster_rename.sh.
Co-authored-by: Junie <junie@jetbrains.com>
- Introduced `knoe.core.prod_config` with production configuration schema and helper methods.
- Added validation logic to enforce required fields and expected formats.
- Implemented in-process API (`ProdConfigApi`) for managing production configs.
- Added test coverage for default config behavior, validation, YAML generation, and API workflow.
- Updated cluster UI layout and tests to integrate production config with new navigation flows.
- Added host inventory and updated service flow to reflect production setup changes.
STABLE BUILD — all 35+ pods Running, zero non-Running pods, CNPG ready=3.
## Architecture changes (Supabase slim-down)
- Removed local Supabase Kong, Storage, MinIO, Imgproxy from deployed stack
- Rewired API routing to shared Kong in kube-system
- Rewired object storage to shared Garage in knoe-system
- Supabase footprint reduced to control-plane only: auth, rest, meta,
analytics, realtime (studio port-forward retained)
- Supabase pods renamed: stripped redundant '-prole-supabase-' prefix via
per-component fullnameOverride (e.g. supabase-auth, supabase-rest, ...)
- Supabase workloads pinned to merlin.prole.org (memory-safe node)
## File changes
- conf/service/prole.cfg
* Replaced all hardcoded /Users/chrisfu/dev/prole paths with ${HOME}/dev/prole
so config is portable when run from myrddin.prole.org or any user home
* SUPABASE_ENABLED=True, SUPABASE_PRIMARY_NODE=merlin.prole.org in
both [Global] and [Optional Features]
* supabase_enabled=true in [Inputs]/init_cluster section
- supabase/helm/render_supabase.py
* Explicitly disabled: kong, storage, minio, imgproxy
* Active components pinned via nodeSelector to merlin.prole.org
* fullnameOverride per component to clean up pod names
* Default primary node changed from pi.prole.org -> merlin.prole.org
- supabase/deploy.sh
* Removed svc/kong port-forward; API via shared Kong in kube-system
* Removed imgproxy/storage from PV/PVC prep and helm --set persistence args
* Fixed kubectl exec missing -i flag for psql heredoc stdin in
setup_knoe_db_for_supabase
* Updated Studio access messaging to reference shared platform Kong
- k8s/prole/knoe-db.yaml
* instances: 3
* Added control-plane/master tolerations for myrddin scheduling
- deploy/opentofu/k3s/manifests/prole/knoe-db.yaml
* instances: 3, added control-plane tolerations (mirrors k8s/prole/)
- k8s/prole/iscsi-pvs.yaml
* Expanded d003-data/d003-wal nodeAffinity to [pi, merlin, myrddin]
to reflect post-OOM-recovery live state
- k8s/registry/deployment.yaml
* nodeSelector restored to myrddin.prole.org (hostPort 5000 owner)
* Added control-plane + master tolerations so pod schedules correctly
- modes/k3s/knoe-db/.version
* Aligned to 140 (image tag 18-140) matching repo-root version
- etc/init_common_services.sh, etc/init_registry.sh
* Minor fixes aligned with slimmed service layout and node targeting
- infrastructure/inventory/host_vars/pi.prole.org.yml
* Updated host vars to reflect post-recovery pi state
- knoe/core/topology.py, knoe/ui/screens/cluster.py,
knoe/ui/screens/cluster_nodes.py
* Topology and UI updates for 3-node cluster representation
- scan/network_description.txt
* Refreshed network scan output post-recovery
- conf/port-mapping.cfg
* Updated port mapping to reflect removal of local Kong/storage forwards
- scripts/cleanup_stale_rs.sh (new)
* Utility script to prune orphaned ReplicaSets after rollouts
## Verified stable state
- CNPG: Cluster in healthy state, ready=3
knoe-db-2 (merlin, primary), knoe-db-3 (pi), knoe-db-5 (myrddin)
- Supabase: 5 pods Running on merlin.prole.org
supabase-auth, supabase-rest, supabase-meta,
supabase-analytics, supabase-realtime
- Monitoring: all prometheus/grafana/alertmanager pods Running on merlin
- Registry: Running on myrddin.prole.org (hostPort 5000)
- Zero non-Running / non-Completed pods across all namespaces
Co-authored-by: Junie <junie@jetbrains.com>
