Commit Graph

25 Commits

Author SHA1 Message Date
chrisfu
96fba3c9bb chore(gitignore): ignore .venv-ansible-winrm/ 2026-06-12 14:25:47 -07:00
chrisfu
998c1bc94d chore: gitignore infrastructure/logs; add ekosystem UUID CNPG plan
- Add infrastructure/logs/ to .gitignore so ansible run logs
  (infrastructure/logs/ansible/*.log) are never accidentally committed
- Track docs/plans/junie/ekosystem-uuid-cnpg-wire.md — Junie brief for
  wiring the ekosystem UUID schema to the CNPG cluster (queue item #13)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-28 08:10:39 -07:00
chrisfu
bb71cf68ea feat(auth): dual IdP for db.prole.org — Google OAuth + Kerberos SPNEGO
knoe-auth (Spring Boot OIDC provider):
- AuthProperties: add google.workspaceDomain and kerberos.servicePrincipal fields
- GoogleOAuthService: validate hd (hosted domain) claim; restrict to configured workspace
- LoginController: /login/google endpoint + SPNEGO negotiation entry point
- PrincipalNormalizer: map Kerberos principal (user@REALM) to knoe-auth user
- application.yml: add spring.security.kerberos and oauth2.client stubs (values
  injected at runtime from env / Kubernetes Secrets)
- knoe-auth-deployment.yaml: mount HTTP keytab Secret; add GOOGLE_PROLE_CLIENT_ID /
  GOOGLE_PROLE_CLIENT_SECRET env from oauth2-proxy-prole-secret
- knoe-auth-http-keytab-secret.example.yaml: example Secret for HTTP/<host> keytab

Kong (init_kong.sh):
- Add db.prole.org route in k3s mode block via oauth2-proxy upstream
- Mode-gate: only registered for k3s, excluded for k3d/k8s

Supabase / oauth2-proxy:
- New supabase/helm/oauth2-proxy Helm chart: gates Supabase Studio at db.prole.org
  with Google OAuth (email-domain=prole.org) + cookie settings for .prole.org domain
- values-k3s.yaml: k3s-specific overrides (upstream service, TLS, cookie domain)
- secret-example.yaml: placeholder for oauth2-proxy-prole-secret

Ansible:
- infrastructure/playbooks/kerberos_trust_setup.yml: automates samba-tool domain
  trust create on myrddin.prole.org for PROLE.LOCAL ↔ PROLE.ORG cross-realm trust

Test:
- GoogleLoginProleOrgTest: verifies hd=prole.org tokens are accepted; hd=other.com rejected

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-03 20:03:09 -07:00
chrisfu
00f0ebec07 Merge claude/crazy-bose-fec256 into main
Bringing the long-running session-feature branch back into main in one
deliberate sweep. The branch carried the cluster work that's been live for
weeks (cross-cluster CNPG metrics, Grafana w/ Google OAuth, supabase
oauth2-proxy, cluster recovery, pg.0.knoe.dev + per-engineer onboarding,
GCS-backed CNPG backups via Workload Identity, the env-contamination
guard, the Junie brief queue, the cnpg-grafana CSRF + memory-request
fixes from today), while main accumulated Junie's parallel knoe-auth
Phase 2 OIDC work (full provider surface: discovery, authorize, token,
userinfo, JWKS, RS256 signing, code exchange, session services).

Key decision: the two branches did COMPETING rebrands off the same
starting point (5ba9b63, 2026-04-27):

  - claude branch (commit b355855, earlier): org.prole.authority.* →
                                              dev.knoe.auth.*
                                              (artifact renamed to
                                              knoe-auth.jar)
  - main (commit 9daa94b, recent): org.prole.authority.* →
                                    dev.knoe.authority.*
                                    (kept "authority" artifact name)

dev.knoe.auth wins: cluster runs from this name, the Maven artifact is
already knoe-auth.jar, and the broader rename is the documented
namespace direction (per ~/.claude/projects/-Users-chrisfu-dev-knoe-db/
memory/MEMORY.md). All of main's recent Phase 2 OIDC content was ported
from authority/src/.../dev/knoe/authority/ into
authority/src/.../dev/knoe/auth/ with package declarations rewritten.

== File-level resolution summary ==

Textual conflicts (4):

  authority/pom.xml
    - Took our artifactId="auth"
    - Took our branch's removal of spring-security-kerberos-client
      (verified: Junie's Phase 2 OIDC code does not import it; the dep
      was already-dead config)

  docs/pipeline-phases.md
    - Took our branch's "Phase 1 not started" status. Main had a
      misplaced " Complete" with a knoe-auth-Phase-1 commit ref
      in the autobuild Phase 1 section — different domain.

  docs/plans/knoe-auth-round-1.md
    - Took our branch's dev.knoe.auth file table (vs main's
      dev.knoe.authority listing). Pure rename mismatch.

  supabase/helm/knoe-supabase/templates/kong/config.yaml
    - Took our branch's onboard route + plain dashboard wiring.
      Main had an oauth2proxy.enabled toggle that put oauth2-proxy as
      a Kong upstream — but the deployed architecture (commit 25f1b2e)
      has oauth2-proxy in FRONT of Kong, not behind. Main's wrapper
      reflected an architecture that was never deployed.
    - Took our branch's removal of basic-auth from dashboard route
      (queue #15 brief still tracks the matching values.yaml /
      kong/deployment.yaml cleanup).

Java tree reconciliation (44 file-pairs):

  20 dual-path source files + 2 dual-path tests
    Body-identical between main's authority/ and our branch's auth/
    after stripping package decls — main's commit 9daa94b was a pure
    rebrand. Took our branch's auth/ version for all 22.

  8 main-only source files (Phase 2 OIDC), ported into auth/:
    web/JwksController.java
    web/OidcAuthorizeController.java
    web/OidcDiscoveryController.java
    web/OidcTokenController.java
    web/OidcUserInfoController.java
    session/OidcCodeService.java
    session/OidcTokenService.java
    session/SessionService.java

  12 main-only test files, ported into auth/:
    HealthControllerTest.java
    enroll/EnrollValueTypesTest.java
    enroll/EnrollmentControllerTest.java
    enroll/TotpServiceTest.java
    kerberos/KadminClientTest.java
    kerberos/KerberosSpnegoResultTest.java
    web/LoginControllerTest.java
    admin/AdminControllerTest.java
    user/PrincipalNormalizerTest.java
    regression/IdentityRegressionTest.java
    session/OidcCodeServiceTest.java
    session/SessionServiceTest.java

  Port mechanics: read main:authority/...<file> via git show, then sed
  rewrite `package dev.knoe.authority` → `package dev.knoe.auth` and
  `import dev.knoe.authority` → `import dev.knoe.auth`. Body content
  unchanged.

  authority/src/main/java/dev/knoe/authority/ — DELETED (duplicate)
  authority/src/test/java/dev/knoe/authority/  — DELETED (duplicate)

== Verification ==

- grep -rln '<<<<<<<' across .java/.md/.yaml/.yml/.sh/.xml/.tpl: clean
- find authority/src -path '*/dev/knoe/authority*': empty (subtree gone)
- grep 'package dev.knoe.authority' across repo: clean
- bash -n install.sh deploy.sh etc/preflight_kubecontext.sh: clean
- git ls-files -u | wc -l: 0 unmerged paths
- helm lint supabase/helm/knoe-supabase: pre-existing failure on
  studioIngress.enabled undefined in values.yaml (introduced by Junie
  on main; unrelated to this merge — flagging as follow-up).

== Followups (carried into TODO ranked queue or noted here) ==

  - helm lint failure: studioIngress block in values.yaml is missing
    enable flag; templates/studio/{ingress,oauth2proxy-deployment,
    oauth2proxy-service}.yaml all reference studioIngress.enabled with
    no default. Pre-existing on main; not introduced by this merge.
  - The five Junie briefs filed on this branch are now reachable from
    main at docs/plans/junie/{02,06,07,13,15}-*.md. Junie can pick them
    up in any order.
  - knoe-auth Phase 2 OIDC source (now at dev.knoe.auth.*) is not yet
    deployed to the cluster. Deployment is its own task.
  - The branch claude/crazy-bose-fec256 stays in place (worktree at
    .claude/worktrees/crazy-bose-fec256 may have ongoing context for
    Claude Code sessions). Safe to delete once next session starts
    cleanly from main.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-01 16:39:10 -07:00
chrisfu
553612bdff chore: purge dead credentials and harden .gitignore
- Delete tracked knoe-db/.prole_user_password (dead secret, rotated
  in ansible since the original commit)
- Remove postgres-password.txt and build/ copy from working tree
- Add explicit ignore rules for *_password, postgres-password*,
  .prole_user_password, *-secret variants

Add docs/plans/customer-deploy-resync.md with the upstream/downstream
migration plan and update CLAUDE.md to reflect prole's role as a
customer deploy of knoe-db.
2026-04-27 13:33:29 -07:00
chrisfu
15df3ecc08 chore: purge ssl/, tfstate, ad-hoc artifacts; harden .gitignore
Follow-up to the ansible excision: remove other artifacts that should
never have been tracked, and extend .gitignore so they do not come back.

Removed:
- ssl/prole/*.key, *.crt, *.srl -- prole.org-era private keys, registry
  and DB CA material. References in knoe/core/env.py (auto-detect path
  at :1488) are best-effort inside try/except, so deletion is safe.
- ssl/prole.ssl.tar -- tarball of the same material.
- deploy/opentofu/k3s/terraform.tfstate -- 114KB committed OpenTofu
  state snapshot with live cluster UIDs. tfstate is never supposed to
  live in git.
- docs/deploy_gke.log, docs/deploy_gke_done.log,
  docs/compressed-history.log -- deploy-time log captures.
- end_time.txt, start_time.txt, start_time_final.txt -- ansible wall-
  clock timing breadcrumbs.
- network_description.txt, network_prompt.txt -- local scratch prompts.
- sidekiq_logs.txt -- transient log capture.
- test_resolve.sh -- ad-hoc debug script.

.gitignore additions:
- *.tfstate, *.tfstate.backup
- /ssl/
- .ai/
- /end_time.txt, /start_time*.txt, /sidekiq_logs.txt,
  /network_description.txt, /network_prompt.txt, /test_resolve.sh,
  /docs/deploy_*.log, /docs/compressed-history.log

Note: the purged SSL keys remain in git history until the deferred
history-scrub pass (task #14). The repo will be created as Private in
git.knoe.dev so this is acceptable for now; it MUST be scrubbed before
any broader visibility change.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-20 19:01:03 -07:00
chrisfu
1aa7c22a3a fix gke secret mananagement 2026-04-20 12:52:03 -07:00
chrisfu
f07fe4fc1c Remove Garage PVC runtime data from source control; gitignore knoe-db/data and var/
knoe-db/data/pvc-* are Garage S3 object store data files from a local k3d dev
cluster. Runtime data has no place in source control. Add knoe-db/data/ and var/
to .gitignore so they can never be accidentally committed again.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-11 16:25:18 -07:00
chrisfu
2e517686a8 Add GCP ncurses TUI to knoe/config.py and config.sh launcher
Consolidates the GCP cluster configuration TUI (formerly etc/config.py)
into knoe/config.py as a __main__ entrypoint, resolving the naming
collision with the root-level installer utilities module. Adds config.sh
as a thin shell launcher at the project root. Also excludes .claude/
worktree directories from git and IDE indexing.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-31 23:15:54 -07:00
chrisfu
5a7a7a5a91 Remove generated values.generated.json from git; add to .gitignore
Co-authored-by: Junie <junie@jetbrains.com>
2026-03-24 14:40:02 -07:00
chrisfu
b09319ffaf feat: pre-provision cluster-scoped CNPG Synology PVs
Add storage ops for deterministic namespace+cluster local PV paths, labels/selectors, host-path preparation, and idempotent reconciliation before CNPG cluster apply.

Wire selector injection and validation into CNPG deploy flow/script, and extend installer tests for provisioning orchestration and failure handling.

Co-authored-by: Junie <junie@jetbrains.com>
2026-03-23 00:32:35 -07:00
chrisfu
0943186d08 installer: add Cluster Nodes screen + update flow
- Add Cluster Nodes spreadsheet screen for host/service placement (Primary Host vs Enabled Here)

- Wire navigation to show Cluster Nodes after Cluster Environment (multi-node, non-k3d)

- Update screen registry/base helpers and related layout/navigation tests

- Refresh port mappings and config fixtures
2026-03-20 16:15:47 -07:00
chrisfu
ecacf01266 Remove legacy secrets, configs, and scripts; introduce prole-auth service and Grafana SSO proxy configuration. 2026-03-20 13:00:36 -07:00
chrisfu
f051d1d42a iscsi/installer: provision merlin d004; fix etc script runtime
- Add merlin iSCSI config for PROLE-DATA-4 mounted at /prole/d004 (xfs, _netdev,noatime)

- Refine iscsi role login/mount flow (device resolution, mkfs-if-missing, UUID fstab, tags)

- Ensure installer run_script stages lib/shell into PROLE_HOME so etc scripts can source common libs; add regression test

- Add init scripts for Forgejo/GitLab; ignore generated conf/prole.cfg

Co-authored-by: Junie <junie@jetbrains.com>
2026-03-16 21:30:52 -07:00
chrisfu
86cf99cf76 milestone(k3s+cnpg): end-to-end deploy run via standalone installer
We can now complete a full K3s deployment run and deliver a working CloudNativePG (CNPG) stack from the standalone installer.\n\nHighlights:\n- Installer scripts hardened for CNPG deploy + recovery templating + rollout control\n- OpenTofu/K3s manifests aligned with in-repo k8s templates for prole-db\n- Supporting init flows updated (OpenBao, monitoring, Kerberos, OpenTofu)

Co-authored-by: Junie <junie@jetbrains.com>
2026-03-03 20:03:15 -08:00
chrisfu
d2efb835b0 Refactor project structure and update initialization scripts
- Moved files from 'prole/' subdirectory to root level or appropriate subdirectories (tests, authority, infrastructure) to flatten the project structure.

- Updated 'install.py' and initialization scripts in 'etc/' to reflect the new directory layout.

- Added 'etc/repair_pipeline.sh' for automated pipeline repairs.

- Updated configuration files including 'conf/prole.cfg' and 'env.sh'.

- Integrated ArgoCD manifests in 'k8s/argocd/'.

- Updated 'prole-app' environment and properties.

- Moved and updated test scripts for better organization and reliability.

- Added 'tests/silent_install_test.sh' for automated installation testing.
2026-02-14 13:44:49 -08:00
chrisfu
87e2f5d385 Checkpoint: Refactor common services and monitoring initialization. Updates: etc/init_monitoring.sh (Grafana naming, legacy cleanup), etc/init_common_services.sh (migration/cleanup logic), added etc/init_registry.sh, updated etc/init_cloudnative_pg.sh (manifest filtering), updated configs, and enhanced tests. Fixed: excluded .ansible logs and added to .gitignore. 2026-02-09 18:50:44 -08:00
chrisfu
f47c18fef7 feat(infrastructure): enhance k3s automation and OpenTofu integration
- Infrastructure:
    - Updated k3s Ansible role with mountpoint preflight checks and better permission management.
    - Automated deployment of prole configuration and port-forwarding scripts to cluster hosts.
    - Added systemd service for managing port forwards on k3s nodes.
    - Added prole-installer service account token automation.
- K8s Manifests:
    - Renamed and added Persistent Volumes in iscsi-pvs.yaml (including OpenBao support).
    - Updated StatefulSets for garage and openbao.
    - Migrated prole-db to CloudNativePG-based configuration.
    - Added comprehensive OpenTofu manifests for cluster deployment.
- Configuration:
    - Added cluster-specific configurations (k3d, k3s-hosts).
    - Added PostgreSQL configuration templates.
    - Updated .gitignore to track the conf/ directory.
- Tools:
    - Updated install.py and port-forwarding scripts.
    - Added render_manifest.py for manifest generation.
2026-02-07 22:53:18 -08:00
chrisfu
42ec9bb7ab Refactor shell scripts for manifest rendering via prole_render_manifest function, introduce dynamic port forwarding configuration, and improve handling for k3d compatibility. 2026-02-06 21:22:26 -08:00
chrisfu
82dc879116 cleanup: remove references to prole/workstation 2026-02-05 14:56:01 -08:00
chrisfu
620ce63190 checkpoint: update infrastructure scripts, manifests, and installer. etc/init_*.sh: added namespace support and OpenBao monitoring configuration; k8s/prole: added grafana-pvc, updated prole-db with S3 region support; install.py: implemented config secret encryption and improved ARM64 platform detection; .gitignore: expanded ignore patterns; prole-net: updated prole-agent binary 2026-02-04 01:37:46 -08:00
chrisfu
9c1f58caf4 Checkpoint: namespace management improvements and prole-db updates. Enhanced namespace management with automated namespace detection and configuration in etc/prole_cfg.sh and etc/init_prole-db.sh. Added namespace reset and recovery scripts: scripts/reset-ns.sh, etc/init_prole-db-reset.sh, and k8s/prole/prole-db-recovery.yaml.tpl. Updated prole-db image versioning to use release files and bumped version to 17.7-041. Improved installer UI and configuration handling in install.py and installer/screen.py. Updated Kerberos and CloudNativePG initialization scripts for better namespace support. Added /prole/backup/ to .gitignore. 2026-01-31 19:22:05 -08:00
chrisfu
e69b5f2008 ansible: replace no_swap role with swap role and enhance K3s setup
- Removed the `no_swap` role and replaced it with the new `swap` role for managing swap configurations.
- Updated the K3s role to dynamically detect the first available wired interface IP and validate it.
- Enhanced K3s installation by adding support for disabling SELinux RPM.
- Adjusted inventory and `site.yml` to align with the updated roles.
- Added `.vault_pass` to `.gitignore`.
- Updated Ansible configuration with result format compatibility for different versions.
2026-01-21 16:16:14 -08:00
chrisfu
96b68c493f remove background files 2026-01-18 16:35:30 -08:00
chrisfu
906392d462 feat(installer): improve UI and add test coverage for core features
- Refactored installer UI with updated canvas rendering, sidebar navigation, and footer buttons.
- Enhanced styling for macOS compatibility and consistent design across controls.
- Added Pytest-based unit tests for `screen.py` and `config.py`.
- Expanded dependency catalog with new tools like `tshark` and `pyshark`.
- Improved error tolerance for background rendering and added placeholders for Kerberos configuration.
2026-01-08 21:51:30 -08:00