Commit Graph

170 Commits

Author SHA1 Message Date
chrisfu
abc458260c ansible: lock in docker_build_host role for the arm64 build lane (gandalf)
Provisions the docker engine on [build_hosts] (gandalf) for the native arm64 daily
knoe-db image build → registry.knoe.dev, run via a remote buildx builder. gandalf is
also a k3s_agent, so the role holds iptables FORWARD=ACCEPT to protect k3s pod
networking (docker's containerd coexists with k3s's on separate sockets). Verified
idempotent: --check against gandalf = ok=5, changed=0.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-21 15:11:08 -07:00
chrisfu
9b0005aa4c fix(samba_ad_dc): bind to LAN IP only so the DC stops registering junk DNS
The AD DC's smb.conf set no `interfaces` / `bind interfaces only`, so Samba
bound to — and samba_dnsupdate registered into DNS — every interface on the
host. On myrddin that meant the Docker bridge (172.17.0.1) and k3s/flannel
CNI addresses (10.42.0.0, 10.42.0.1) were published as A records for both the
prole.org apex and `myrddin`, alongside the real 10.0.0.3. Clients then
round-robined onto unroutable addresses, producing the long-standing
"resolves, then doesn't" internal DNS flakiness.

Confine Samba to loopback + the LAN service IP:

  interfaces = lo 10.0.0.3
  bind interfaces only = yes

This scopes both service binding and DNS self-registration to the real
address, so the junk records stop being (re)created on restart.

  - smb.conf.j2: emit the two directives, gated on bind-interfaces-only being
    enabled AND a non-loopback IP being present (empty -> directives omitted,
    never binds loopback-only by accident).
  - defaults: samba_ad_dc_lan_ip ("" by default), samba_ad_dc_bind_interfaces_only
    (true), samba_ad_dc_interfaces (lo + lan_ip), all documented.
  - tasks: assert samba_ad_dc_lan_ip is non-empty before deploying smb.conf
    when bind-interfaces-only is on, so a missing value fails fast instead of
    rendering the DC unreachable.
  - group_vars/ad_dc: set samba_ad_dc_lan_ip=10.0.0.3 (myrddin's LAN address).

Deploying notifies the existing Restart samba-ad-dc handler. Pre-existing junk
records must be deleted once by hand; they will not be re-registered after the
restart. Template rendering verified for both the set and empty-IP cases.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-10 00:43:28 -07:00
chrisfu
0ae0ac4a8a fix(samba_reverse_dns): inline reverse-zone list so it survives tag filtering
`Create reverse DNS zones if missing` looped over `samba_reverse_zones`, a fact
built by a separate `set_fact` task that carried no tags. Under a tag-filtered
run (e.g. --tags samba_reverse_dns) that set_fact was skipped, so the variable
was undefined and the play failed:

    TASK [samba_reverse_dns : Create reverse DNS zones if missing]
    'samba_reverse_zones' is undefined

A set_fact must carry the same tags as every task that consumes it. Rather than
re-add tags (which breaks again under any other tag combination), compute the
zone list inline in the loop and drop the now-dead set_fact. The role is now
correct under any tag selection. lan_reverse_zone and k3s_reverse_zones are
defined in group_vars/all/vars.yml, so they are always available.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-10 00:35:29 -07:00
chrisfu
c0c43bcf6a fix(samba_dns): strip zone suffix correctly so internal A/CNAME/PTR records are created
Forward A/CNAME and reverse PTR records reported changed=0 / ok but never
resolved by short name on the Samba AD DC (myrddin.prole.org). Root cause was
regex escaping inside YAML folded block scalars (">-"): unlike double-quoted
scalars, block scalars do NOT process backslash escapes, so a pattern written
as '\\.' reached Jinja as a literal backslash + any-char and never matched.

Consequences:
  - regex_replace() never stripped the zone suffix, so records were created
    under their full FQDN (e.g. a record literally named "sg2428lp.prole.org"
    inside zone "prole.org"). `samba-tool dns query ... prole.org sg2428lp A`
    then returns WERR_DNS_ERROR_NAME_DOES_NOT_EXIST.
  - regex_findall() of existing values always returned [], so stale records
    were never pruned and idempotency only survived via RECORD_ALREADY_EXISTS.

The leading/trailing whitespace from the folded scalar was a red herring: the
command module's shlex tokenization collapses it, so a single-token name
survived intact. The escaping was the actual defect.

Fix:
  - Convert the templated set_fact expressions from ">-" block scalars to
    double-quoted single-line scalars (one consistent escaping convention,
    matching the already-working samba_target_zone line; also removes the
    stray whitespace).
  - Add an assertion that the computed record name is a clean, non-empty,
    whitespace-free token.
  - Add a post-apply verification re-query + assert that the expected
    value(s) are actually present, turning a silent no-op into a hard failure.

Applies to roles/samba_dns (ensure_a.yml, ensure_cname.yml) and
roles/samba_reverse_dns (ensure_ptr.yml).

Verified end-to-end through ansible-playbook against a stateful fake
samba-tool: correct short names (sg2428lp, git, @, registry), stale-value
removal, internal.prole.org zone routing, idempotent re-runs (changed=0),
and the verify-assert failing loudly when an add silently does not persist.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-10 00:32:00 -07:00
chrisfu
76522eace2 chore(vault): add WinRM passwords for morgoth (chrisfu) and fairyland (minecraft)
vault_winrm_password_morgoth and vault_winrm_password_fairyland stored for
Ansible WinRM auth to Windows GPU workstations.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-02 21:28:15 -07:00
chrisfu
26790b8ced fix(pihole): use Pi-hole 6 dns.hosts for knoe.dev records
Pi-hole 6 uses pihole-FTL --config dns.hosts as the authoritative local DNS
store. dnsmasq address= directives still work for new domains (e.g. mcp.0.knoe.dev)
but Pi-hole may serve stale upstream cache over them for previously-resolved
domains (e.g. auth.0.knoe.dev was cached from old DNS before the fix).

dns.hosts entries always win, idempotent approach: strip old *.0.knoe.dev entries,
append new ones, update via pihole-FTL --config + pihole reloaddns.

auth.0.knoe.dev: 34.120.221.5 (wrong) → 34.36.71.141 (knoe-auth GCE LB) ✓

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-02 15:00:40 -07:00
chrisfu
3877cbfd38 feat(pihole): add knoe.dev GKE service records for LAN split-horizon DNS
Adds dnsmasq address= directives for all active knoe.dev GKE endpoints
so LAN clients (morgana, Pi nodes) resolve them without hitting external DNS.

Records added:
  mcp.0.knoe.dev   → 8.232.93.14    (knoe-mcp GCE LB)
  auth.0.knoe.dev  → 34.36.71.141   (knoe-auth GCE LB)
  api.0.knoe.dev   → 34.120.221.5   (supabase-kong GCE LB)
  db.0.knoe.dev    → 34.120.221.5   (supabase-kong GCE LB)
  app.0.knoe.dev   → 34.120.221.5   (supabase-kong GCE LB)

auth.0.knoe.dev was previously resolving to 34.120.221.5 (wrong — supabase LB)
causing the knoe-auth managed cert to be stuck Provisioning for 20 days.
Correct IP is 34.36.71.141. Also update auth.0.knoe.dev in external DNS
(name.com) to unblock the GCP managed cert.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-02 14:39:47 -07:00
chrisfu
11ba0c6f57 feat(winrm): wire Ansible WinRM for morgoth+fairyland Windows GPU hosts
- ansible.sh: export OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES (fixes macOS
  fork safety abort when pywinrm loads ObjC frameworks before fork())
- hosts.ini: add winrm scheme=http port=5985, transport=ntlm for Windows hosts;
  remove hardcoded ansible_user (now per-host)
- group_vars/workstations_windows/vars.yml: created; WinRM connection vars
  (ansible_user/password resolved per-host)
- host_vars/morgoth.prole.org.yml: ansible_user=chrisfu + vault ref
- host_vars/fairyland.prole.org.yml: ansible_user=minecraft + vault ref
- windows_ollama_models.yml: fix PowerShell ollama show check — wrap in
  try/catch so ErrorActionPreference=Stop does not abort on model-not-found

Result: mxbai-embed-large-2k (num_ctx=2048) created on both hosts.
Idempotent — re-running only recreates if Modelfile hash changes.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-02 02:16:05 -07:00
chrisfu
f0d3a85fbc Avoid fact gathering for Windows Ollama playbook 2026-06-01 23:24:11 -07:00
chrisfu
e5fe9dc2a1 Disable Unix become for Windows inventory hosts 2026-06-01 23:08:40 -07:00
chrisfu
3b1014ee4d Disable sudo become for Windows Ollama playbook 2026-06-01 23:03:49 -07:00
chrisfu
5f60ede5d7 Add Windows Ollama model configuration playbook 2026-06-01 22:04:11 -07:00
chrisfu
5077e1335d feat(gitea): SPNEGO Kerberos SSO for git.prole.org
Three-part fix to make browser and curl SPNEGO auth work end-to-end:

1. spnego-proxy: issue WWW-Authenticate: Negotiate challenge when no
   Authorization header is present so Chrome (with AuthServerAllowlist)
   and curl --negotiate automatically present Kerberos tokens. Previously
   the proxy only validated tokens if the client proactively sent them.
   Pass-through preserved for non-Negotiate schemes (Basic/token) so
   git CLI users with PATs continue to work via Gitea own auth.

2. gitea_spnego_keytab.yml: new Ansible playbook that provisions the
   gitea-http AD account (AES-only, msDS-SupportedEncryptionTypes=24),
   registers SPN HTTP/git.prole.org, resets the password to derive fresh
   AES keys, exports the domain keytab, and rekeys it to principal name
   HTTP/git.prole.org@PROLE.ORG that gokrb5 needs for keytab lookup.
   Key lesson: samba-tool exportkeytab --principal=HTTP/... returns empty;
   must export full domain keytab and rekey in Python.

3. init_gitea.sh: add setup_gitea_spnego() calling the Ansible playbook
   in k3s mode as part of the standard deploy flow, with inline notes
   on every non-obvious constraint discovered during this work.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-28 05:19:59 -07:00
chrisfu
b245593b0c fix(ansible): workstation install script + Makefile target for local Macs
ansible_become_ask_pass in group_vars does not reliably trigger an
interactive sudo prompt for ansible_connection=local — Ansible cannot
prompt per-host mid-play. The fix is a wrapper script that passes
-K / --ask-become-pass before any play execution.

- infrastructure/bin/install_workstation.sh: auto-detects FQDN, adds
  --ask-become-pass, passes extra args through. Run from repo root:
    ./infrastructure/bin/install_workstation.sh
    make workstation
- Makefile: add 'workstation' target pointing to the script
- group_vars/workstations.yml: drop ansible_become_ask_pass (does not
  work), keep ansible_pipelining=false (needed for SSH workstations
  with sudo passwords — pipelining replaces stdin, breaking sudo -S)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-27 21:15:49 -07:00
chrisfu
d945d88b5d fix(ansible): workstations group vars — become_ask_pass + no pipelining
Personal Macs require a sudo password; the global become_ask_pass=False
and pipelining=True combination silently swallows the prompt and fails.

- ansible_become_ask_pass: true  → Ansible prompts for sudo password
- ansible_pipelining: false      → sudo can read stdin for the password;
                                   pipelining replaces stdin with a pipe
                                   which sudo treats as non-interactive and
                                   refuses to ask for a password at all.

Applies to [workstations] group (mac + linux). Has no effect on
[workstations_windows] (WinRM auth is separate from become).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-27 21:10:26 -07:00
chrisfu
9571f42568 fix(ansible): wire vault_password_file to repo-root .vault_pass
Both ansible.cfg files lacked vault_password_file, causing
'Attempting to decrypt but no vault secrets found' when running any
playbook that auto-loads the encrypted group_vars/all/ files.

- ansible.cfg (root): vault_password_file = .vault_pass
- infrastructure/ansible.cfg: vault_password_file = ../.vault_pass

.vault_pass is gitignored. Each machine needs its own copy.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-27 21:05:49 -07:00
chrisfu
cf33342500 feat(prole): bootstrap knoe-auth on k3s; tenant onboarding; cluster stabilisation
knoe-auth (prole.org k3s):
- Fix CNPG manifest drift: remove spec.backup.pluginConfiguration (CNPG 1.28 only),
  switch spec.certificates from serverTLSSecret to serverAltDNSNames
- Apply knoe-auth Round 1 schema + GRANTs manually (postInitSQL had never run on live cluster)
- Fix OIDC signing key generator: base64(DER) not base64(PEM) — OidcTokenService
  does Base64.decode() → PKCS8EncodedKeySpec which requires raw DER bytes
- Add OIDC controllers: authorize, token, userinfo, jwks, discovery
- Add prole Spring profile: cookieDomain, emailDomain, Kerberos config
- Add secret example templates: knoe-db-user, knoe-auth-oidc-signing, knoe-auth-google-prole
- Kong configmap: scope knoe-auth route to /auth prefix only

Tenant onboarding:
- Add etc/onboard_tenant.sh: provision/apply/rotate/status workflow backed by 1Password
  vaults; types: 'enterprise' (own Kerberos + domain) and 'tenant' (hosted, initContainer KDC)
- Provision 'Knoe Tenant - prole.org' vault; apply all 7 k8s secrets to knoe-system
- init_knoe_auth.sh: add explicit GRANT + ALTER DEFAULT PRIVILEGES for knoe role

Cluster stabilisation:
- gitea: roll back 14-day stuck rollout (RWO PVC + maxSurge=100% deadlock);
  patch deployment strategy to Recreate
- supabase: create supabase_admin role, _supabase db, _analytics schema, _realtime schema
  in CNPG — analytics and realtime had never connected since Helm install day 1
- knoe-db barman ObjectStore: add GCS-backed objectstore manifest + scheduled backup

Infrastructure:
- gandalf host_vars: k3s registry config
- pi host_vars: clean up stale entries
- knoe-db schemas: ekosystem.sql, ekosystem_objects.sql
- init_prole_app.sql: prole app DB initialisation

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-26 00:50:37 -07:00
chrisfu
3ada16c875 feat(infrastructure): add Pi-hole DNS flush playbook and logs dir
infrastructure/playbooks/pihole_flush_dns.yml — Ansible playbook to
flush Pi-hole DNS cache across inventory hosts.

infrastructure/logs/ — log directory for Ansible run output.

Co-authored-by: Junie <junie@jetbrains.com>
2026-05-23 21:31:52 -07:00
chrisfu
b5f17ffa72 fix(samba_dns): update vault with correct AD Administrator password
vault_samba_dns_admin_pass was stale — samba-tool commands were failing
with NT_STATUS_LOGON_FAILURE. Updated to match current AD Administrator
credential so the samba_dns role can manage records end-to-end.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-14 23:58:09 -07:00
chrisfu
a2a82bc495 fix(samba_dns): use admin credentials instead of machine Kerberos (-P)
All samba-tool dns commands were using -P (machine account Kerberos) which
silently failed with no error — tasks reported changed=0 but records were
never written. Switch to -U Administrator --password={{ samba_dns_admin_pass }}
which uses the vault-protected admin credentials that were already defined
but never wired up.

Also fix regex patterns in record parsers: samba-tool output uses `A: IP`
and `PTR: fqdn` format, not `A IP` / `PTR fqdn` (space-separated), so
updated regex_findall patterns to match `TYPE:\s+value`.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-14 23:15:13 -07:00
chrisfu
e014fd5b71 feat(k3s): add pg.prole.org as CNPG postgres endpoint with split-horizon DNS
- knoe-db.yaml: switch to CNPG-managed TLS cert with serverAltDNSNames
  (pg.prole.org + knoe-db-rw cluster service) — removes static serverTLSSecret/serverCASecret
- dns.yml: add pg.prole.org A record to prole_k3s_dns_records (10.0.0.3, 10.0.0.6)
  for Ansible-managed split-horizon DNS via Samba AD DC
- k3s.cfg: align KNOE_HOME paths to ~/dev/prole, add PROLE_KDC_* vars, remove
  hardcoded KUBECTL_CONTEXT (kubeconfig current-context is authoritative)
- prod.cfg: add PROLE_KDC_STORAGE_CLASS = prole-iscsi
- onepassword.py: skip vault check gracefully when no 1Password session active (non-TTY)
- CLAUDE.md: document production postgres connection string and DNS/CA cert ops

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-14 22:46:46 -07:00
chrisfu
ad1ecedb5e kerberos_trust_setup: pin trust account to RC4 only (msDS-Supported... 4)
The matching knoe-db change (commit `ff7546d`) re-keys the MIT-side
cross-realm krbtgts with `-e arcfour-hmac:normal` to dodge the
AES-salt mismatch between Samba and MIT. For the trust to actually
carry traffic, the Samba side must also offer only RC4 on the
trust account — otherwise the client picks AES, the two sides
derive different keys from the same password (different salt
conventions), and TGT decryption fails.

Change `msDS-SupportedEncryptionTypes` from 28 (RC4+AES128+AES256)
to 4 (RC4 only). RC4 has no salt, so keys derive from the password
alone and both sides converge.

Updated docstring + summary debug print to reflect the new value
and the why.

Tracked alongside the Junie brief at
~/dev/knoe-db/docs/plans/junie/kdc-trust-reset-repeatable.md.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-11 00:29:01 -07:00
chrisfu
19d1f136de ansible: fix klist principal form in trust-setup summary print
klist shows the Kerberos principal (slash form
krbtgt/KNOE.LOCAL@PROLE.ORG), not the sAMAccountName underscore form
(krbtgt_KNOE.LOCAL). Update the summary debug task accordingly and
split the kvno hint into two stages (the cross-realm TGT first, then
a service ticket) so the user can isolate which step fails when
chasing salt/enctype issues.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-10 22:37:54 -07:00
chrisfu
78eef6fbd9 ansible: hoist ldb-tools install to top of play
The apt install task was positioned after the user-create step, so a
failure in create (or anywhere upstream) meant ldb-tools never got
installed. Move the apt task to the very top of the play, before the
cluster-secret lookups, so it always runs first on every invocation.

Drop the duplicate apt task that previously sat just before the
ldbmodify locator.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-10 22:35:18 -07:00
chrisfu
4be9fbcafe ansible: idempotency check now uses underscore account name
The _trust_exists probe was still grepping samba-tool user list for
"krbtgt/KNOE.LOCAL" (slash form), but the actual sAMAccountName is
"krbtgt_KNOE.LOCAL" (underscore form, since slash is invalid in
sAMAccountName). _trust_exists was therefore always false, the create
step retried against the existing account on every re-run, and Samba
rejected it as a name collision.

Match against samba_account_name directly so re-runs route through
the setpassword branch instead.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-10 22:32:49 -07:00
chrisfu
52ab023ae2 ansible: install ldb-tools package (provides ldbmodify)
myrddin doesn't have ldbmodify installed — it's in the ldb-tools
Debian package, separate from samba-tool. Add an apt task ahead of
the locator so the package is in place before we try to use it.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-10 22:20:26 -07:00
chrisfu
c45671283b ansible: locate ldbmodify binary explicitly (not on sudo PATH)
ldbmodify is part of the ldb-tools Debian package but is not in
root's default PATH under sudo on the Samba DC, so the user-attribute
modify step failed with "ldbmodify: command not found".

Add a locator task that probes the conventional install paths
(/usr/bin, /usr/sbin, /usr/local/bin, /opt/samba/bin) and falls back
to a depth-limited find under /usr and /opt. The downstream
ldbmodify invocation now uses the absolute path resolved here.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-10 22:19:38 -07:00
chrisfu
3485046dbd ansible: temp-disable Samba password complexity for trust account write
samba-tool user create rejected the cluster-generated trust_shared_password
with "0000052D: Constraint violation - check_password_restrictions: the
password does not meet the complexity criteria". The trust password is
high-entropy random bytes from init_kdc.sh and may not satisfy AD's
"3 of 4 character categories" rule.

Wrap the create + setpassword steps in a block that:
  1. reads the current `samba-tool domain passwordsettings show` state
  2. flips complexity off if it was on
  3. does the user write
  4. always: restores complexity to its prior on/off state

The restore is in an `always:` clause so a failure inside the write
block does not leave the domain policy weakened.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-10 22:09:41 -07:00
chrisfu
aa541af3f5 ansible: use krbtgt_KNOE.LOCAL sAMAccountName + UPN for trust account
samba-tool user create rejected the sAMAccountName "krbtgt/KNOE.LOCAL"
with "samldb: sAMAccountName contains invalid '/' character". AD/Samba
disallow "/" in sAMAccountName even though Kerberos principal names
require it (krbtgt/REMOTE@LOCAL).

Switch storage name to "krbtgt_KNOE.LOCAL" and add an ldbmodify pass
that sets the canonical Kerberos identity on the same object:

  userPrincipalName     = krbtgt/KNOE.LOCAL@PROLE.ORG
  servicePrincipalName  = krbtgt/KNOE.LOCAL
  msDS-SupportedEncryptionTypes = 28  (RC4+AES128+AES256)

Samba KDC resolves principals by UPN/SPN, so a TGS-REQ for
krbtgt/KNOE.LOCAL@PROLE.ORG will hit this account.

Note: key-salt parity with the MIT side is NOT guaranteed yet. Samba's
default salt for AES keys is REALM+UPN; MIT's default for cross-realm
krbtgt is REALM+"krbtgt"+REMOTE. If kvno fails with "decrypt integrity
check failed" we'll add a keytab-export/import step in a follow-up
rather than try to coerce Samba's salt at creation time.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-10 22:02:55 -07:00
chrisfu
6740e3dcc5 ansible: guard krbtgt smoke probe against --check mode failure
In --check mode the user-create step is skipped (it's mutating), so
the subsequent `samba-tool user show krbtgt/KNOE.LOCAL` probe fails
with rc=1 on a first dry-run because the user doesn't yet exist. Add
the same guarded failed_when used on the trust validate/show tasks
earlier: only fail when NOT (check_mode AND trust didn't pre-exist).

Real runs still fail hard on rc!=0.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-10 21:57:51 -07:00
chrisfu
555da4acfc ansible: rewrite kerberos_trust_setup for MIT KDC trust
samba-tool domain trust create --type=external only works against
another writeable AD-style DC; against an MIT KDC it fails with
"Failed to find a writeable DC for domain 'KNOE.LOCAL'". Our peer at
svc/auth in knoe-system is a plain Kerberos KDC, so we cannot use that
path.

Replace the trust create/validate/show steps with the supported
Samba-AD <-> MIT-KDC approach: create the inter-realm TGT principal as
an ordinary user account in Samba whose sAMAccountName is
"krbtgt/KNOE.LOCAL", with its password synced to the cluster Secret
knoe-system/knoe-kdc-secrets/trust_shared_password (which the MIT side
already keys against). Steps:

  1. samba-tool user list                            (idempotency probe)
  2. samba-tool user create krbtgt/KNOE.LOCAL ...    (when missing)
  3. samba-tool user setpassword krbtgt/KNOE.LOCAL   (when present, to
     re-sync after a cluster Secret rotation)
  4. samba-tool user setexpiry --noexpiry
  5. ldbmodify msDS-SupportedEncryptionTypes=28      (RC4+AES128+AES256
     to match what init_kdc.sh sets on the MIT side)
  6. samba-tool user show                            (smoke probe)
  7. debug task prints the manual kvno smoke-test command

Cluster-secret lookups and krb5.conf [realms]/[domain_realm] edits are
unchanged - those parts were correct.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-10 15:54:24 -07:00
chrisfu
227f49042c ansible: opt read-only kubectl/samba-tool tasks into --check mode
Ansible --check mode auto-skips command/shell tasks unless they
explicitly opt in via check_mode:false. The two cluster-state lookups
(svc/auth ClusterIP and knoe-kdc-secrets/trust_shared_password) were
silently skipped during dry-run, leaving trust_kdc_ip empty and tripping
the assertion immediately.

Add check_mode:false to the five read-only tasks so a --check run can
still resolve cluster state and report what would change:
  - Resolve trust_kdc_ip from cluster
  - Resolve trust_shared_password from knoe-kdc-secrets
  - samba-tool domain trust list  (idempotency probe)
  - samba-tool domain trust validate
  - samba-tool domain trust show  (smoke print)

The mutating create step (samba-tool domain trust create) keeps the
default behavior - skipped in check mode. validate/show gain a guarded
failed_when so a check-mode dry-run on a host where the trust does not
yet exist does not fail (the create was skipped, so a non-zero rc is
expected). Real runs still fail hard on rc!=0.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-10 01:51:30 -07:00
chrisfu
5cece408bd ansible: switch kerberos_trust_setup to KNOE.LOCAL + vault wiring
The cross-realm trust playbook was written for the pre-rebrand realm name
PROLE.LOCAL, which no longer exists in the in-cluster KDC — the canonical
realm is now KNOE.LOCAL. The Samba AD trust was therefore never actually
established, leaving chrisfu@KNOE.LOCAL→chrisfu@PROLE.ORG service ticket
flows blocked (and blocking pg_oauth / db.prole.org Kerberos work).

Changes:
- Realm: PROLE.LOCAL → KNOE.LOCAL across all task names, vars, and the
  krb5.conf [realms] / [domain_realm] blocks added on myrddin.
- samba_admin_password now resolves from the Ansible vault var
  vault_samba_dns_admin_pass (group_vars/ad_dc/vault.yml) by default,
  with SAMBA_ADMIN_PASSWORD env and -e overrides preserved for CI.
- trust_shared_password auto-resolves from the in-cluster Secret
  knoe-system/knoe-kdc-secrets (key trust_shared_password) when not
  passed explicitly — same Secret init_kdc.sh writes.
- Added [domain_realm] mapping for *.knoe.local → KNOE.LOCAL so Samba
  can resolve in-cluster service principals.
- Added a final `samba-tool domain trust show` smoke step so a successful
  run prints the visible trust state for log review.

Run:
  ANSIBLE_VAULT_PASSWORD_FILE=$PWD/.vault_pass   ansible-playbook infrastructure/playbooks/kerberos_trust_setup.yml

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-10 00:26:31 -07:00
chrisfu
48def19637 fix(inventory): specify ssh key for zinfandel — non-default key name 2026-05-09 22:42:26 -07:00
chrisfu
9020eae91e fix(ansible): remove become from Windows play — connect as admin directly
win_regedit writes HKLM keys without privilege escalation when
ansible_user is already a local/domain Administrator. become:true
with --ask-become-pass was overriding the playbook's runas method.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-09 22:36:48 -07:00
chrisfu
67b483bcf2 fix(ansible): split workstation playbook into two plays for Unix vs Windows
A single play with become:true uses sudo by default, which Ansible's
Windows exec wrapper rejects immediately at gather_facts. Split into:
- Play 1: hosts workstations,!workstations_windows — macOS/Linux, sudo
- Play 2: hosts workstations_windows — Windows, become_method:runas

Also set morgana to ansible_connection=local (running Ansible from
morgana itself, so SSH self-lookup fails DNS resolution).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-09 22:33:51 -07:00
chrisfu
5cd22c53d9 feat(inventory): add all 4 workstations; add Windows SPNEGO policy support
Add morgana and zinfandel (macOS) and morgoth + fairyland (Windows)
to the workstations inventory.

Extend workstation_kerberos.yml to handle Windows targets:
- Chrome and Edge AuthServerAllowlist set via win_regedit under
  HKLM\SOFTWARE\Policies\{Google\Chrome,Microsoft\Edge}
- krb5.conf and plist tasks skip on Windows (uses built-in SSPI/AD)
- Windows hosts in [workstations_windows] group with WinRM/NTLM
  transport; switch to kerberos transport after domain join

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-09 22:21:21 -07:00
chrisfu
7b782afef4 chore: restore vault secrets added on myrddin 2026-05-09 21:48:54 -07:00
chrisfu
7b391ab5ed feat(onboarding): scalable Kerberos SSO onboarding — service account auth, Ansible workstation policy
Problem: 'defaults write' is not an onboarding path, it's a support
incident waiting to happen. Browser SPNEGO must be deployed via managed
policy at machine provisioning time, not per-user.

Changes:
  etc/init_knoe_users.sh
  - _op_ensure_auth(): tries OP_SERVICE_ACCOUNT_TOKEN env, k8s secret
    'op-service-account-token', existing interactive session, and op-run
    inheritance — in that order. Gives actionable instructions when none
    work, including the correct 'op run --' invocation documented in k3s.cfg.
  - gitea_ensure_password(): calls _op_ensure_auth() at entry; eliminates
    the silent 'op not authenticated' failure path.
  - Script header: documents 'op run -- bash etc/init_knoe_users.sh' as
    the intended invocation for admin's laptop.
  - 'Next steps' output: replaces 'defaults write' with reference to
    workstation_kerberos.yml Ansible playbook.

  infrastructure/playbooks/workstation_kerberos.yml (new)
  - Deploys /etc/krb5.conf (PROLE.ORG realm, myrddin KDC) to all managed
    endpoints.
  - Deploys Chrome + Edge managed policy (AuthServerAllowlist) on macOS
    and Linux — no per-user browser configuration ever required.
  - Idempotent; run during laptop provisioning or re-run at any time.
  - Targets 'workstations' Ansible group.

  infrastructure/inventory/hosts.ini
  - Adds [workstations] group with example entries and onboarding notes.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-09 20:10:32 -07:00
chrisfu
1892e137c1 feat(infra): replace GitLab with Gitea on git.prole.org
- Remove GitLab hostname/storage config from k3s.cfg; GitLab is gone.
- Update GITEA_HOSTNAME git-internal → git.prole.org
- Fix MONITORING_STORAGE_CLASS local-path → merlin-local-iscsi-prometheus
- Kong: route git.prole.org → gitea-http.gitea:3000
- Add svc-knoe-ingress.yaml as source-of-truth for the prole ingress (was
  missing; live object named svc-knoe-ingress, routes git.prole.org through
  Traefik → Kong → Gitea)
- DNS: add git.prole.org A record → 73.15.20.166 (public front-door)
- monitoring/kps-values-k3s.yaml: fix storage class for Prometheus/Alertmanager

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-07 01:25:32 -04:00
chrisfu
52f4053718 fix(acme): replace fragile key-file regex selector with rejectattr
selectattr with 'match' regex was returning empty sequence on this Ansible
version. Since each install entry has exactly two items (fullchain.cer +
key), rejectattr the fullchain to get the key — no regex needed.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-06 15:57:57 -04:00
chrisfu
e0c96f47d4 fix(acme): use correct mixed-case Namecom_Username/Token variable names
The dns_namecom.sh plugin uses Namecom_Username and Namecom_Token (not
NAMECOM_*). Fix the environment dict, the SAVED_ keys written to
account.conf, and add a cleanup task to remove the stale all-caps entries
already present on hosts from previous runs.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-06 15:35:20 -04:00
chrisfu
18f2ce2167 fix(acme): restore no_log and switch to command+environment for credential safety
Re-adds no_log: true removed in 5e4e17e and switches the issue task from
ansible.builtin.shell with inline VAR=value to ansible.builtin.command with
an environment: dict. This prevents credentials appearing in Ansible failure
output and avoids the inline env var being clobbered when acme.sh sources
account.conf internally before the dns_namecom plugin runs.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-06 15:27:44 -04:00
chrisfu
5e4e17ec60 debug: temporarily remove no_log from issue task to see acme.sh output 2026-05-06 15:15:47 -04:00
chrisfu
372496eea2 fix(acme): use shell inline env vars for issue; bypass sudo env_reset
Ansible command module + become does not reliably pass environment:
vars through sudo env_reset. Use ansible.builtin.shell with POSIX
inline assignment (VAR=val cmd) so credentials are set inside the
sudo-spawned shell subprocess, never touching the sudo env barrier.
account.conf writing retained for cron renewal.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-06 15:14:12 -04:00
chrisfu
5ddb76930f fix(acme): use SAVED_ prefix for namecom creds in account.conf
_readaccountconf_mutable checks SAVED_NAMECOM_Username first; if that
key lookup exits 0 with empty output it short-circuits and never reads
the bare NAMECOM_Username key. Write the SAVED_ prefix that acme.sh
uses internally when persisting credentials.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-06 15:10:13 -04:00
chrisfu
76b641842e fix(acme): write namecom creds to account.conf instead of env vars
sudo env_reset strips the environment: key through become. Write
NAMECOM_Username and NAMECOM_Token to acme.sh's account.conf directly,
which dns_namecom reads as a fallback via _readaccountconf_mutable.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-06 14:57:28 -04:00
chrisfu
d268693b2a fix(acme): pin --server letsencrypt on issue to avoid ZeroSSL fallback
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-06 14:55:03 -04:00
chrisfu
86990af9c8 feat(certs): add acme.sh DNS-01 role and cert-manager playbook for LE
- New acme role: installs acme.sh, issues certs for myrddin/merlin/gandalf
  via DNS-01 (name.com API), installs cert files to /etc/ssl/certs/prole/,
  sets up renewal cron. rc=2 (already valid) treated as ok.
- New playbooks/acme.yml: runs acme role on myrddin (issues all 3 host certs)
- New playbooks/certmgr.yml: cert-manager and letsencrypt-prod ClusterIssuer
  (HTTP-01/Traefik) for external endpoints (db/svc/api.prole.org)
- Add vault references for name.com API creds to group_vars/all/vars.yml
- Add prole_ssl_src_dir and prole_ssl_files to myrddin/merlin/gandalf host_vars

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-06 14:09:28 -04:00
chrisfu
296bcd15a5 fix(dns,ssl): add gandalf A record; make prole_ssl resilient to missing certs
- Add gandalf.prole.org → 10.0.0.7 to prole_dns_records and
  prole_internal_a_records in dns.yml (was missing, causing nslookup
  to fail on myrddin)
- prole_ssl: stat-check each source file on the controller before
  copying; emit a warning and skip rather than hard-failing when certs
  have not yet been placed in ssl/prole/

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-05 20:22:22 -04:00