Commit Graph

16 Commits

Author SHA1 Message Date
chrisfu
d114801758 feat: wire ekosystem UUID system into CNPG manifests (Tasks 1, 3, 4)
Task 1 — ConfigMap + CNPG wiring:
- Add k8s/knoe/knoe-ekosystem-sql.yaml: ConfigMap embedding ekosystem.sql
  and ekosystem_objects.sql for CNPG postInitApplicationSQLRefs
- Add scripts/gen-ekosystem-configmap.py: generation script to keep the
  ConfigMap in sync with knoe-db/schema/ekosystem*.sql source files
- Add Makefile target: make k8s/knoe/knoe-ekosystem-sql.yaml
- Wire postInitApplicationSQLRefs into all three CNPG cluster manifests:
    k8s/knoe/knoe-db.yaml (k3s / prole-service-context production)
    deploy/gcp/gke/knoe-db.yaml (GKE)
    deploy/opentofu/k3s/manifests/knoe/knoe-db.yaml (OpenTofu k3s)
- Add knoe-ekosystem-sql.yaml to k8s/knoe/kustomization.yaml

Task 3 — Python counterpart utility:
- Add knoe/ekosystem.py: thread-safe EkosystemID generator matching the
  PostgreSQL bit layout [49:ts_ms|12:tenant|10:shard|11:seq], with
  decode() and can_access() helpers
- Add tests/test_ekosystem.py: 23 tests covering base36 encoding,
  round-trips, thread safety, can_access, and the spec round-trip assertion

Task 4 — knoe.user ekosystem_uuid column:
- Add ALTER TABLE knoe.user ADD COLUMN IF NOT EXISTS ekosystem_uuid text UNIQUE
  to postInitSQL in all three CNPG manifests

Task 2 (register prole tenant) requires a live DB connection — manual step.
Task 5 (LDAP/AD reconciler) is design-only per spec.

Co-authored-by: Junie <junie@jetbrains.com>
2026-05-30 00:26:48 -07:00
chrisfu
b245593b0c fix(ansible): workstation install script + Makefile target for local Macs
ansible_become_ask_pass in group_vars does not reliably trigger an
interactive sudo prompt for ansible_connection=local — Ansible cannot
prompt per-host mid-play. The fix is a wrapper script that passes
-K / --ask-become-pass before any play execution.

- infrastructure/bin/install_workstation.sh: auto-detects FQDN, adds
  --ask-become-pass, passes extra args through. Run from repo root:
    ./infrastructure/bin/install_workstation.sh
    make workstation
- Makefile: add 'workstation' target pointing to the script
- group_vars/workstations.yml: drop ansible_become_ask_pass (does not
  work), keep ansible_pipelining=false (needed for SSH workstations
  with sudo passwords — pipelining replaces stdin, breaking sudo -S)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-27 21:15:49 -07:00
chrisfu
903f84f200 feat: ship Junie #3 (image rename) + Phase 2 OIDC GKE deploy + k3d chrisfu seed
Three independent lines of work landing in one commit because they were all
on disk together end-of-session and the cross-file edits (TODO, knoe-system,
junie/README) interleave cleanly:

1. **Queue #3 — image rename `knoe-authority` → `knoe-auth`** (Junie).
   Closes drift R6. The Maven artifact has been `knoe-auth.jar` since
   commit b355855; the deploy manifests now match.
     authority/Dockerfile.app                                         (NEW)
     deploy/gcp/gke/knoe-auth-deployment.yaml                         (3 image tags renamed)
     deploy/opentofu/k3s/manifests/knoe/knoe-auth-deployment.yaml     (2 image tags renamed)
     Makefile                                                         (docker-build-auth +
                                                                       docker-push-auth + REGISTRY/KNOE_AUTH_VERSION
                                                                       defaults)
     docs/plans/junie/03-image-rename-knoe-authority-to-knoe-auth.md  (brief, kept as design record)

2. **Phase 2 OIDC provider — GKE deploy** (Junie). Source landed via the
   merge that brought claude/crazy-bose-fec256 back; the k3d sandbox
   shipped earlier today (commit 93157b0). This commit completes the GKE
   path: Kong route `/auth` → knoe-auth, default values added, signing
   key wired via `knoe-auth-oidc` secret per the brief.
     supabase/helm/knoe-supabase/templates/kong/config.yaml           (Kong route /auth, gated
                                                                       on .Values.knoeAuth.enabled)
     supabase/helm/knoe-supabase/values.yaml                          (knoeAuth.enabled=false default;
                                                                       studioIngress block — also
                                                                       fixes the helm lint issue noted
                                                                       in earlier commit 03bb731)
     docs/plans/junie/phase2-oidc-gke-deploy.md                       (brief, kept as design record)

3. **k3d dev-user seed: `chrisfu` + `knoe_developer`** (Claude). Closes
   "I want auth as chrisfu@knoey.com to my local cnpg database" for the
   local dev loop. `etc/init_knoe_auth.sh --mode k3d` now runs
   `seed_dev_users_k3d()` after the schema bootstrap, creating:
     - `knoe_developer` group role with R/W on `knoe`+`public` (mirrors
       the GKE production layout from docs/db-access.md; was hand-rolled
       in production per the 2026-04-30 onboarding work, never baked into
       postInitTemplateSQL).
     - `chrisfu` LOGIN role with password `chrisfu-dev` (idempotent —
       resets on every `make k3d-knoe-up`, so the rebuild loop is
       deterministic). Granted into `knoe_developer`.
   From the host with port-forward up:
     PGPASSWORD=chrisfu-dev psql "postgresql://chrisfu@localhost:5432/knoe-db?sslmode=require"

Cross-cutting doc updates:
  docs/TODO.md            — Phase 2 GKE entry removed from "In progress"
                            (now "(none — all items shipped or paused)");
                            queue #3 + reality table R6 removed; pg_oauth
                            paused note flipped to "OIDC issuer now
                            deployed to GKE — pg_oauth can resume"; Done
                            section gets new entries for #3, Phase 2 GKE,
                            and chrisfu seed.
  docs/knoe-system.md     — Phase 2 GKE row → Shipped; pg_oauth → Ready;
                            knoe-auth-deployment.yaml note "knoe-auth:latest"
                            instead of "knoe-authority:latest"; "One-time
                            cluster setup" mentions the chrisfu seed.
  docs/local-dev-knoe-auth.md — "Verify psql connectivity" rewritten to
                            connect as chrisfu (was `knoe`); new "Reset /
                            rebuild loop" section explaining idempotency.
  docs/plans/junie/README.md — two new rows under Shipped: brief 03 and
                            phase2-oidc-gke-deploy.

Verification (the bits I ran locally):
  - bash -n etc/init_knoe_auth.sh                                     OK
  - The seed SQL is idempotent (DO blocks with EXISTS checks +
    ALTER ROLE on the password reset path)
  - The Kong /auth route is gated on .Values.knoeAuth.enabled — default
    false, no behavior change until someone flips it on per Junie's brief

Out of scope for this commit:
  - Actually flipping knoeAuth.enabled=true on the live GKE chart and
    rolling out — Junie's brief covers the runbook; needs the
    knoe-auth-oidc K8s secret populated from 1Password first
  - Rebuilding + pushing the new knoe-auth:latest image — `make
    docker-push-auth` is wired, just hasn't been run yet
  - Round 1.5 OpenBao transit-key encryption (still queued)

Co-authored-by: Junie <junie@jetbrains.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-02 13:07:40 -07:00
chrisfu
5d3600845d feat(k3d): laptop dev loop for knoe-auth — CNPG + KDC + port-forward
Brings up the smallest k3d-resident stack that lets a host-side knoe-auth
(run via `mvn spring-boot:run` or IntelliJ) iterate against real Postgres
+ Kerberos. Closes Phase 1 of the k3d-gke-mirror plan (docs/plans/k3d-gke-mirror.md).

Scope:
  - k8s/knoe/knoe-kdc-{configmap,deployment,service,pvc,init-job}.yaml
    NEW; standalone KDC, realm KNOE.LOCAL (distinct from KNOE.DEV).
  - etc/init_knoe_auth.sh: --mode k3d flag added; swaps realm + skips
    GCP-specific steps. GKE behavior unchanged when flag absent.
  - Makefile: k3d-knoe-up, k3d-knoe-pf, k3d-knoe-down (delegate to
    scripts/k3d-knoe-{up,pf,down}.sh).
  - scripts/k3d-knoe-{up,pf,down,smoke}.sh NEW; up = full bring-up,
    pf = three port-forwards (5432/88/464) + JDBC URL + ^C cleanup,
    down = teardown, smoke = sanity check.
  - etc/krb5.local.conf NEW; checked-in libdefaults+realms config
    pointing at localhost:88. udp_preference_limit=1 to dodge
    kubectl port-forward UDP flakiness on macOS.
  - docs/local-dev-knoe-auth.md NEW; one-time setup + daily loop +
    IntelliJ run config.
  - docs/knoe-system.md NEW; unified reference for the knoe-auth
    service (GKE deployment + k3d dev loop + schema overview +
    source map + open work items).

Verified per the brief's Definition of done: fresh-clone laptop can
`make k3d-knoe-up` + `make k3d-knoe-pf` + `mvn -pl authority spring-boot:run`
and hit /health, /.well-known/openid-configuration in <8 minutes.

Out of scope (parent plan docs/plans/k3d-gke-mirror.md §6):
  - SPNEGO from host browsers (Phase 2)
  - knoe-auth-as-pod / image build/load (Phase 3)
  - Supabase stack on k3d (Phase 4)
  - OidcCodeService DB persistence (separate track)

docs/plans/junie/README.md — k3d brief moved from Active to Shipped.
docs/TODO.md — In-progress now empty; Phase 2 pg_oauth notes that the
local dev loop is in place so it can resume.

Closes Phase 1; Phase 2+ briefs filed as needed.

Co-authored-by: Junie <junie@jetbrains.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-02 11:47:09 -07:00
chrisfu
00f0ebec07 Merge claude/crazy-bose-fec256 into main
Bringing the long-running session-feature branch back into main in one
deliberate sweep. The branch carried the cluster work that's been live for
weeks (cross-cluster CNPG metrics, Grafana w/ Google OAuth, supabase
oauth2-proxy, cluster recovery, pg.0.knoe.dev + per-engineer onboarding,
GCS-backed CNPG backups via Workload Identity, the env-contamination
guard, the Junie brief queue, the cnpg-grafana CSRF + memory-request
fixes from today), while main accumulated Junie's parallel knoe-auth
Phase 2 OIDC work (full provider surface: discovery, authorize, token,
userinfo, JWKS, RS256 signing, code exchange, session services).

Key decision: the two branches did COMPETING rebrands off the same
starting point (5ba9b63, 2026-04-27):

  - claude branch (commit b355855, earlier): org.prole.authority.* →
                                              dev.knoe.auth.*
                                              (artifact renamed to
                                              knoe-auth.jar)
  - main (commit 9daa94b, recent): org.prole.authority.* →
                                    dev.knoe.authority.*
                                    (kept "authority" artifact name)

dev.knoe.auth wins: cluster runs from this name, the Maven artifact is
already knoe-auth.jar, and the broader rename is the documented
namespace direction (per ~/.claude/projects/-Users-chrisfu-dev-knoe-db/
memory/MEMORY.md). All of main's recent Phase 2 OIDC content was ported
from authority/src/.../dev/knoe/authority/ into
authority/src/.../dev/knoe/auth/ with package declarations rewritten.

== File-level resolution summary ==

Textual conflicts (4):

  authority/pom.xml
    - Took our artifactId="auth"
    - Took our branch's removal of spring-security-kerberos-client
      (verified: Junie's Phase 2 OIDC code does not import it; the dep
      was already-dead config)

  docs/pipeline-phases.md
    - Took our branch's "Phase 1 not started" status. Main had a
      misplaced " Complete" with a knoe-auth-Phase-1 commit ref
      in the autobuild Phase 1 section — different domain.

  docs/plans/knoe-auth-round-1.md
    - Took our branch's dev.knoe.auth file table (vs main's
      dev.knoe.authority listing). Pure rename mismatch.

  supabase/helm/knoe-supabase/templates/kong/config.yaml
    - Took our branch's onboard route + plain dashboard wiring.
      Main had an oauth2proxy.enabled toggle that put oauth2-proxy as
      a Kong upstream — but the deployed architecture (commit 25f1b2e)
      has oauth2-proxy in FRONT of Kong, not behind. Main's wrapper
      reflected an architecture that was never deployed.
    - Took our branch's removal of basic-auth from dashboard route
      (queue #15 brief still tracks the matching values.yaml /
      kong/deployment.yaml cleanup).

Java tree reconciliation (44 file-pairs):

  20 dual-path source files + 2 dual-path tests
    Body-identical between main's authority/ and our branch's auth/
    after stripping package decls — main's commit 9daa94b was a pure
    rebrand. Took our branch's auth/ version for all 22.

  8 main-only source files (Phase 2 OIDC), ported into auth/:
    web/JwksController.java
    web/OidcAuthorizeController.java
    web/OidcDiscoveryController.java
    web/OidcTokenController.java
    web/OidcUserInfoController.java
    session/OidcCodeService.java
    session/OidcTokenService.java
    session/SessionService.java

  12 main-only test files, ported into auth/:
    HealthControllerTest.java
    enroll/EnrollValueTypesTest.java
    enroll/EnrollmentControllerTest.java
    enroll/TotpServiceTest.java
    kerberos/KadminClientTest.java
    kerberos/KerberosSpnegoResultTest.java
    web/LoginControllerTest.java
    admin/AdminControllerTest.java
    user/PrincipalNormalizerTest.java
    regression/IdentityRegressionTest.java
    session/OidcCodeServiceTest.java
    session/SessionServiceTest.java

  Port mechanics: read main:authority/...<file> via git show, then sed
  rewrite `package dev.knoe.authority` → `package dev.knoe.auth` and
  `import dev.knoe.authority` → `import dev.knoe.auth`. Body content
  unchanged.

  authority/src/main/java/dev/knoe/authority/ — DELETED (duplicate)
  authority/src/test/java/dev/knoe/authority/  — DELETED (duplicate)

== Verification ==

- grep -rln '<<<<<<<' across .java/.md/.yaml/.yml/.sh/.xml/.tpl: clean
- find authority/src -path '*/dev/knoe/authority*': empty (subtree gone)
- grep 'package dev.knoe.authority' across repo: clean
- bash -n install.sh deploy.sh etc/preflight_kubecontext.sh: clean
- git ls-files -u | wc -l: 0 unmerged paths
- helm lint supabase/helm/knoe-supabase: pre-existing failure on
  studioIngress.enabled undefined in values.yaml (introduced by Junie
  on main; unrelated to this merge — flagging as follow-up).

== Followups (carried into TODO ranked queue or noted here) ==

  - helm lint failure: studioIngress block in values.yaml is missing
    enable flag; templates/studio/{ingress,oauth2proxy-deployment,
    oauth2proxy-service}.yaml all reference studioIngress.enabled with
    no default. Pre-existing on main; not introduced by this merge.
  - The five Junie briefs filed on this branch are now reachable from
    main at docs/plans/junie/{02,06,07,13,15}-*.md. Junie can pick them
    up in any order.
  - knoe-auth Phase 2 OIDC source (now at dev.knoe.auth.*) is not yet
    deployed to the cluster. Deployment is its own task.
  - The branch claude/crazy-bose-fec256 stays in place (worktree at
    .claude/worktrees/crazy-bose-fec256 may have ongoing context for
    Claude Code sessions). Safe to delete once next session starts
    cleanly from main.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-01 16:39:10 -07:00
chrisfu
55b6a6aff3 Complete rebranding from prole to knoe and fix macOS application identity. Bulk renamed 'prole' to 'knoe' across code, scripts, and manifests. Updated Makefile with 'knoe', 'build', and 'start' targets. Implemented macOS Application Bundle (.app) support for correct identity. Fixed macOS 'Python' process name to 'Knoe.DB Installer' via Objective-C bridge. Standardized application name to 'Knoe.DB Installer' across all interfaces.
Co-authored-by: Junie <junie@jetbrains.com>
2026-04-22 15:08:35 -07:00
chrisfu
2477d23bab Rename prole.sh to knoe.sh and implement start/stop/restart for min mode
Co-authored-by: Junie <junie@jetbrains.com>
2026-04-22 10:02:29 -07:00
chrisfu
f44ce32a27 Rebrand and consolidate authentication and scanning systems. Consolidate Kerberos authentication system into authority/src and rename ProleAuthApplication to KnoeAuthApplication. Remove deprecated prole/, prole-app/, prole-mssql-db/, and prole-tools-app/ directories. Migrate and rename prole-net/prole-agent to scan/network-agent. Update install.sh, Makefile, and documentation to use Knoe branding and new scan paths. Rebrand configuration properties and URLs to knoe.dev and svc.knoe.dev.
Co-authored-by: Junie <junie@jetbrains.com>
2026-04-22 09:28:14 -07:00
chrisfu
4ee2b259c9 Checkpoint: rename installer to knoe + harden db build context
- Add build-context helper to copy Docker context safely (ignore runtime data, keep symlinks)

- Update UI and core actions to use ~/.prole/build and shared copy helper

- Add/adjust tests and scripts; introduce knoe ops helpers and update manifests

Co-authored-by: Junie <junie@jetbrains.com>
2026-03-22 01:45:21 -07:00
chrisfu
f20df131df Fix Percona PG 18 build and update installer configuration.
- Dockerfile: Resolved interactive tzdata prompts with debconf and fixed Percona package names (percona-pg-stat-monitor18 and percona-postgresql-contrib).

- Configuration: Updated prole.cfg with specific namespace (prole-hq0-db0), user (chrisfu), and encrypted secrets.

- Installer: Enhanced initialization scripts (init_openbao.sh, init_certmgr.sh) and updated service layers for cluster deployment.

- Tests: Added coverage for dependent image collection in installer core.

- Version: Bumped prole-db version to 77.

Co-authored-by: Junie <junie@jetbrains.com>
2026-02-28 23:11:32 -08:00
chrisfu
5e0a1bda85 feat: Add GitOps (Gitea) and Supabase integration, plus database options
- Makefile: Added 'init' and 'deploy' targets for k3s parity and Gitea staging.

- OpenTofu: Fixed namespace handling in k3s main.tf to prevent metadata overwrites.

- UI: Added 'GitOps' and 'Database Options' configuration screens.

- Core: Enhanced monitoring, milestones, and environment handling for new services.

- Supabase: Integrated full Helm chart and manifest rendering logic.

- Gitea: Added deployment scripts and GitOps sync support.

- Database: Added Percona/Postgres Dockerfile templates and improved TDE scripts.

- Tests: Added coverage for new UI screens and navigation flows.
2026-02-26 18:32:15 -08:00
chrisfu
dafed53810 Enhance build system with 'make test' and fix installer test debris. Added 'make test' to Makefile with 'pyconv' (black) integration and coverage summary. Reformatted codebase with black. Fixed 'install.py' test bug where MagicMock objects created directory debris by improving mocking and patching in 'tests/test_navigation.py', 'tests/test_service_layer_navigation.py', and 'tests/test_install_logic.py'. 2026-02-25 14:55:05 -08:00
chrisfu
1573bb59c2 Add prole.spec and extend OpenTofu k3s configuration
- Include new `prole.spec` for build configurations and dependencies.
- Add Terraform state handling for OpenTofu in `k3s` cluster.
- Provision multiple Kubernetes resources in `prole-db` namespace: namespace, services, ConfigMaps, StatefulSets, Ingress rules, and PersistentVolumes.
- Integrate deployment and configuration enhancements for `garage`, `prole`, and related components.
2026-02-24 21:47:02 -08:00
chrisfu
51c91595b9 Refactor infrastructure, installer, and deployment configs: expanded CloudNativePG and PostgREST init scripts, updated ArgoCD/OpenTofu/K8s manifests (ingress, kustomization, postgrest, prole-db), added installer config helpers and revised cluster UI, updated prole.cfg and prole.sh, simplified Makefile, added env helper tests, removed stale spec/log files, added prole/ directory 2026-02-23 21:19:19 -08:00
chrisfu
11746f40e9 chore: remove legacy Analysis-00.toc file from build directory
Deleted outdated and unused Analysis-00.toc file to clean up the project structure.
2026-01-19 23:35:07 -08:00
chrisfu
cbfe930b78 feat: add ncurses interface, build system, and embedded resources
Major feature additions and infrastructure improvements for the Prole
Database Installer, enabling command-line operation and packaged binary
distribution.

## Ncurses Terminal Interface

- Add installer/ncurses_ui.py: UI primitives (CursesWindow, TerminalConsole,
  NavFooter, InputField, Checkbox)
- Add installer/ncurses_installer.py: Complete terminal UI with all 11 screens
- Implement same screen flow as GUI (welcome, deps, network scan, env setup,
  kerberos, password, build, cluster, scripts, deploy, installer creation)
- Add keyboard navigation (arrows, hjkl, vim-style)
- Support both GUI and ncurses modes in single binary

## Automatic Display Detection

- Add has_display() function to detect GUI availability
- Auto-select GUI if display available, ncurses otherwise
- Add --gui and --no-gui command-line flags
- Fallback to ncurses on GUI failure

## Build System and Packaging

- Add Makefile with targets: build, package, clean, test, install
- Add scripts/generate_spec.py: PyInstaller spec generator
- Add installer.spec: PyInstaller configuration
- Automatic PNG to ICNS icon conversion
- Create self-contained macOS .app bundle with embedded icon
- Support both Intel (x86_64) and Apple Silicon (arm64)

## Embedded Resources

- Add get_resource_path() helper for PyInstaller compatibility
- Embed all images (proleIcon.png, proleLogo.png, proleLogoSepia.png)
- Embed prole-net/prole-scan binary (6.8 MB universal binary)
- Embed prole-app/dist/Prole Tools.app (12 MB app bundle)
- Embed prole-db/ Docker build context

## Writable Directory Fixes

- Create ~/.prole/build/prole-db/ for Docker builds (fixes read-only _MEIPASS)
- Create ~/.prole/scan/ for network scan output (fixes API call failures)
- Copy build context to writable location before Docker operations
- Run prole-scan from writable working directory

## Documentation

- docs/build-system.md: Complete build system guide
- docs/ncurses-installer.md: Ncurses interface documentation
- docs/RELEASE-NOTES.md: Feature overview and release notes
- docs/IMAGE-RESOURCES.md: Image resource management
- docs/EMBEDDED-RESOURCES.md: Binary and app bundle embedding
- docs/DOCKER-BUILD-FIX.md: Docker build hang solution
- docs/PROLE-HOME-DIRECTORY.md: ~/.prole directory structure
- BUILD.md: Quick build reference

## Key Changes

install.py:
- Add get_resource_path() for embedded resource resolution
- Update image paths to use get_resource_path()
- Update Docker build to use ~/.prole/build/prole-db/
- Update network scan to use ~/.prole/scan/
- Add display detection and mode selection
- Add --gui and --no-gui argument parsing

## Testing

All features tested and verified:
- Ncurses interface navigation
- Display auto-detection
- Resource path resolution
- Docker build from package
- Network scan from package
- Icon conversion and embedding

Package size: ~50-100 MB (includes Python runtime, all resources)
Disk usage: ~/.prole/ uses ~2-6 MB

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-01-19 23:10:37 -08:00