Commit Graph

24 Commits

Author SHA1 Message Date
chrisfu
48def19637 fix(inventory): specify ssh key for zinfandel — non-default key name 2026-05-09 22:42:26 -07:00
chrisfu
67b483bcf2 fix(ansible): split workstation playbook into two plays for Unix vs Windows
A single play with become:true uses sudo by default, which Ansible's
Windows exec wrapper rejects immediately at gather_facts. Split into:
- Play 1: hosts workstations,!workstations_windows — macOS/Linux, sudo
- Play 2: hosts workstations_windows — Windows, become_method:runas

Also set morgana to ansible_connection=local (running Ansible from
morgana itself, so SSH self-lookup fails DNS resolution).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-09 22:33:51 -07:00
chrisfu
5cd22c53d9 feat(inventory): add all 4 workstations; add Windows SPNEGO policy support
Add morgana and zinfandel (macOS) and morgoth + fairyland (Windows)
to the workstations inventory.

Extend workstation_kerberos.yml to handle Windows targets:
- Chrome and Edge AuthServerAllowlist set via win_regedit under
  HKLM\SOFTWARE\Policies\{Google\Chrome,Microsoft\Edge}
- krb5.conf and plist tasks skip on Windows (uses built-in SSPI/AD)
- Windows hosts in [workstations_windows] group with WinRM/NTLM
  transport; switch to kerberos transport after domain join

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-09 22:21:21 -07:00
chrisfu
7b391ab5ed feat(onboarding): scalable Kerberos SSO onboarding — service account auth, Ansible workstation policy
Problem: 'defaults write' is not an onboarding path, it's a support
incident waiting to happen. Browser SPNEGO must be deployed via managed
policy at machine provisioning time, not per-user.

Changes:
  etc/init_knoe_users.sh
  - _op_ensure_auth(): tries OP_SERVICE_ACCOUNT_TOKEN env, k8s secret
    'op-service-account-token', existing interactive session, and op-run
    inheritance — in that order. Gives actionable instructions when none
    work, including the correct 'op run --' invocation documented in k3s.cfg.
  - gitea_ensure_password(): calls _op_ensure_auth() at entry; eliminates
    the silent 'op not authenticated' failure path.
  - Script header: documents 'op run -- bash etc/init_knoe_users.sh' as
    the intended invocation for admin's laptop.
  - 'Next steps' output: replaces 'defaults write' with reference to
    workstation_kerberos.yml Ansible playbook.

  infrastructure/playbooks/workstation_kerberos.yml (new)
  - Deploys /etc/krb5.conf (PROLE.ORG realm, myrddin KDC) to all managed
    endpoints.
  - Deploys Chrome + Edge managed policy (AuthServerAllowlist) on macOS
    and Linux — no per-user browser configuration ever required.
  - Idempotent; run during laptop provisioning or re-run at any time.
  - Targets 'workstations' Ansible group.

  infrastructure/inventory/hosts.ini
  - Adds [workstations] group with example entries and onboarding notes.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-09 20:10:32 -07:00
chrisfu
e4478f20ab feat(tailscale): add Ansible role and playbook for merlin + gandalf
Installs Tailscale on k3s agents via the Debian stable apt repo using
the modern signed-by keyring approach. Auth key stored in vault.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-05 18:57:05 -04:00
chrisfu
5ff46a0f3a infrastructure: restore ansible configuration and refactor samba_dns for internal zone - Restored the 'infrastructure' directory and root-level 'ansible.sh', 'ansible.cfg' scripts. - Refactored 'samba_dns' role to dynamically handle 'prole.org' and 'internal.prole.org' DNS zones. - Switched 'samba-tool' commands to use machine account authentication (-P) in 'samba_dns' and 'samba_reverse_dns'. - Updated AD DC inventory variables to use 127.0.0.1 and correct admin principal. - Added tags to 'samba_dns' tasks for better target execution. - Updated IDE project configuration for knoe-db.
Co-authored-by: Junie <junie@jetbrains.com>
2026-04-28 00:05:03 -07:00
chrisfu
5a769f273b chore: excise prole.org ansible infrastructure for knoe-db fork
knoe-db deploys cluster-native (mode k3d for local dev, k8s for the
shared company cluster, k3s still supported for individual homelab
collaborators via conf/k3s.cfg). The ansible-managed distributed cluster
of prole.org machines is no longer the target topology, and the Ansible
inventory in this tree targets only the original author's specific hosts
(merlin/gandalf/myrddin/pi/raspberry/retropie.prole.org). It is neither
portable nor useful for the new deployment shape.

Removed:
- infrastructure/inventory/  (hosts.ini + 7 *.prole.org host_vars)
- infrastructure/playbooks/  (11 playbooks, all hardcoded to *.prole.org)
- infrastructure/roles/      (20 roles: pihole, samba AD DC, mariadb,
                              netplan, iscsi, prole_ssl, etc. -- all
                              tied to the bare-metal prole.org topology)
- infrastructure/setup/      (cr_ansible_user.sh, cr_samba_family_users.sh,
                              filesystem.txt -- prole.org bootstrap)
- infrastructure/deployments/svc-check-helm/
                              (legacy; etc/init_kong.sh already tears
                               down svc-check resources on every run)
- infrastructure/ansible.cfg (references deleted inventory)
- ansible.cfg, ansible.sh, ansible_min.cfg, ansible_recovery.log  (root;
                              orphaned without inventory)
- vault_backup/              (ansible-vault encrypted prole.org secrets
                              backup: ad_dc_vault.yml, all_prole_vault.yml,
                              iscsi_vault.yml)
- tools/                     (dashboard.sh, ghcr_docker_login.sh,
                              k3s-cluster-rename.sh, manage-node.sh,
                              export_mariadb_synology.sh,
                              prepare_mariadb_usb.sh, and a dumped
                              synology.prole.org mariadb .sql file --
                              all prole.org-era deployment helpers)

Kept: scripts/ (cluster ops), bin/ (prole-env.sh, prole-kpf.sh dev
shims), conf/k3s.cfg / conf/k3d.cfg (still supported modes; any
*.prole.org defaults in those files are legitimate mode fallbacks for
collaborators who keep running their own k3s/k3d setups). Test
fixtures referencing *.prole.org hostnames are left as-is (regression
data).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-20 18:53:03 -07:00
chrisfu
b8a55888be checkpoint: update GKE deployment docs and infra changes
Summarize CLAUDE.md updates:

- Document both live GKE clusters as e2-standard-2 x3 in us-west3 and note stale app default in code.

- Call out DB cluster naming mismatch between conf (knoe-dev-cnpg-0) and code default (knoe-cnpg-0).

- Add SSD quota guidance: 300GB fully consumed by CNPG, so non-CNPG PVCs must use standard/pd-standard.

- Refresh reset script reference and quota section details to match current runtime state.

Co-authored-by: Junie <junie@jetbrains.com>
2026-04-18 07:18:10 -07:00
chrisfu
eb9430df9d fix(gitlab,infra): ARM64 RPi service cluster – GitLab deploy in gitlab ns on gandalf
Namespace & routing
- milestones.py: GitOpsMilestone now resolves namespace from
  gitops.gitlab_namespace (new) → Global.GITLAB_NAMESPACE → 'gitlab'
  hardcoded; never falls through to gitops.namespace (was 'gitea')
- prole.cfg: add gitops.gitlab_namespace=gitlab + GITLAB_NAMESPACE=gitlab
- init_gitlab.sh: NAMESPACE defaults to gitlab, NODE_SELECTOR blanked so
  only gitaly+minio are node-pinned; GITOPS_NAMESPACE fallback removed

GitLab on ARM64 RPi (16 KB kernel pages)
- init_gitlab.sh: DaemonSet compiles jemalloc-5.3.0 with --with-lg-page=14
  (glibc/Ubuntu) on every node; LD_PRELOAD injected per Ruby component
- Minio: quay.io 2022 image (ARM64); configure init container replaced
  with ARM64 alpine that writes credential files; MINIO_ROOT_USER/PASSWORD
  injected directly into main container env via secretKeyRef
- Minio buckets auto-created post-deploy (registry, lfs, artifacts, etc.)
- webservice/sidekiq: replicaCount=1, reduced memory (1500M/800M),
  liveness probe initialDelaySeconds=3600 (Rails loads 25-40min on RPi)
- allowedHosts set as flat string list (chart 9.x default is list-of-maps
  which breaks URI initializer in 7_gitlab_http.rb)
- gitaly+minio always pinned to gandalf (local PV); other workloads spread

Storage
- Static PVs created for gitaly (50Gi) + minio (10Gi) on synology d005
- Synology dirs created before PVs; bucket creation idempotent

Redis (shared for GitLab KAS)
- init_redis.sh: persistence disabled (no dynamic provisioner); Redis used
  as pub/sub broker only

Infrastructure / pi.prole.org
- Removed pi.prole.org from [k3s_agents] – dedicated pihole node, OOM
- host_vars: k3s_enabled=false, k3s_state=absent (storage preserved)
- New playbook: infrastructure/playbooks/disable_pi_k3s.yml (drain + disable)
- monitoring.py: node-exporter DaemonSet excludes pi.prole.org
- init_monitoring.sh: pi.prole.org excluded from node-exporter affinity
- kong-deployment.yaml: affinity rule prevents scheduling on pi (pihole owns 80/443)

Co-authored-by: Junie <junie@jetbrains.com>
2026-04-02 20:15:11 -07:00
chrisfu
d02aadae89 Add node management workflow and installer config persistence updates
- add tools/manage-node.sh for k3s node add/remove automation with inventory reconciliation

- include kubectl drain flag compatibility and --ssh-host targeting for uninstall workflows

- persist installer cfg/env values and update deploy/database/cfg UI handling

- refresh related inventory/network artifacts and expand installer/database test coverage

Co-authored-by: Junie <junie@jetbrains.com>
2026-03-29 11:22:33 -07:00
chrisfu
fb7993de1d feat: add production config schema, validation, and API with tests
- Introduced `knoe.core.prod_config` with production configuration schema and helper methods.
- Added validation logic to enforce required fields and expected formats.
- Implemented in-process API (`ProdConfigApi`) for managing production configs.
- Added test coverage for default config behavior, validation, YAML generation, and API workflow.
- Updated cluster UI layout and tests to integrate production config with new navigation flows.
- Added host inventory and updated service flow to reflect production setup changes.
2026-03-28 10:43:25 -07:00
chrisfu
598eae4633 Enable TLS for k3s registry and deploy SSL certs
- Add ssl_hosts group and prole_ssl role to deploy registry cert/key to /etc/ssl/certs/prole

- Configure k8s registry deployment to serve HTTPS using mounted host certs

- Switch k3s/containerd registry mirror config to https:// and prefer HTTPS checks with HTTP fallback

Co-authored-by: Junie <junie@jetbrains.com>
2026-03-10 00:58:08 -07:00
chrisfu
436c6214df checkpoint: k3s agent config + node labels + registry optimizations
- k3s: render server-only config keys only for servers; add regression test; make guardrail test non-sudo

- k3s/site: add post-provision node labeling tasks + playbook tier; extend inventory node label mapping

- registry: default namespace to common-services and skip redundant image pushes by probing registry manifests (installer + init scripts)
2026-03-07 03:14:26 -08:00
chrisfu
d2295f4c2b ansible: add K3s datastore export/import, improve iSCSI handling, and migrate merlin to MariaDB primary
- Added tasks and playbooks for K3s datastore export/import using MariaDB Tools role, with associated tests and defaults.
- Enhanced iSCSI role to support mkfs-once logic and safer re-initialization of block storage.
- Migrated iSCSI-backed Rancher data from retropie to merlin.prole.org.
- Updated k3s roles/playbooks to relax Rancher storage preflight checks, supporting PARTUUID-based mounts.
- Adjusted Samba AD NetBIOS name derivation to use uppercase short hostname by default.
- Incremented prole DB version to 104, updated generated prole.cfg, inventory, and recovery manifest templates.
2026-03-06 14:25:13 -08:00
chrisfu
cf6d79f505 ansible: fix netplan_static YAML, dashboard conky pkg, mariadb vault vars
- netplan_static: avoid Jinja whitespace/indent YAML breakage with trim_blocks; set /etc/netplan perms to 0600; add render+YAML-parse test

- dashboard: install concrete conky provider (conky-all) and add defaults test; wire role into site.yml; document dashboard service usage

- mariadb: add mariadb parent group for group_vars scope; add group defaults deriving datastore password from vault; add vault entry; add vars resolution test; remove host overrides

- misc: update port-forward mappings, generated prole.cfg, and bump prole-db version

Co-authored-by: Junie <junie@jetbrains.com>
2026-03-05 23:02:42 -08:00
chrisfu
78a16f47e2 ansible: add merlin k3s host + USB-backed MariaDB bring-up
- Add merlin.prole.org (10.0.0.6) to inventory (k3s_hosts, mariadb_primary)

- Relax k3s preflight: /var/lib/rancher no longer requires iSCSI, only blocks SD-backed storage

- Add netplan static IP role (NetworkManager) and MariaDB primary/replica roles + site.yml ordering

- Add/upgrade USB prep tooling (prepare_mariadb_usb.sh + generated setup.sh + fstab-by-LABEL)

Co-authored-by: Junie <junie@jetbrains.com>
2026-03-05 19:17:40 -08:00
chrisfu
3d29ab70f2 feat(k3s): enhance k3s install and factory reset reliability. Implement dynamic token discovery, serialized cluster setup with 'serial: 1', readiness checks, and image pre-staging for more robust cluster bootstrapping. 2026-02-16 15:04:04 -08:00
chrisfu
035d63090c Fix Ansible YAML parsing errors and standardize host_vars. Removed invalid '---' document separators from host_vars files, merged redundant myrddin config, and renamed host_vars to use .yml extension. Verified with successful ansible-playbook --syntax-check. 2026-01-25 23:36:25 -08:00
chrisfu
20d408ff36 add iscsi role and vault 2026-01-25 15:31:33 -08:00
chrisfu
e69b5f2008 ansible: replace no_swap role with swap role and enhance K3s setup
- Removed the `no_swap` role and replaced it with the new `swap` role for managing swap configurations.
- Updated the K3s role to dynamically detect the first available wired interface IP and validate it.
- Enhanced K3s installation by adding support for disabling SELinux RPM.
- Adjusted inventory and `site.yml` to align with the updated roles.
- Added `.vault_pass` to `.gitignore`.
- Updated Ansible configuration with result format compatibility for different versions.
2026-01-21 16:16:14 -08:00
chrisfu
1a69097699 fix hostname typo 2026-01-21 10:38:39 -08:00
chrisfu
772a36294e ansible: configure K3s cluster and update inventory
- Added `k3s_hosts` group to the inventory and associated host entries.
- Updated `site.yml` to include roles for disabling swap and setting up the K3s cluster.
2026-01-21 10:23:05 -08:00
chrisfu
a6cc1173e9 ansible: update inventory and configure rsyslog for Linux hosts
- Add `Linux_Hosts` group in inventory with child nodes.
- Update `site.yml` to apply `rsyslog` role to `Linux_Hosts` with privilege escalation.
2026-01-20 22:16:32 -08:00
chrisfu
c4156f58d0 ansible starts here 2026-01-18 20:02:41 -08:00