chrisfu/prole
1
0
Fork 0
mirror of https://github.com/dredx/prole.git synced 2026-09-23 11:03:59 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity
00f0ebec07
prole/AGENTS.md
chrisfu 00f0ebec07 Merge claude/crazy-bose-fec256 into main 
Bringing the long-running session-feature branch back into main in one
deliberate sweep. The branch carried the cluster work that's been live for
weeks (cross-cluster CNPG metrics, Grafana w/ Google OAuth, supabase
oauth2-proxy, cluster recovery, pg.0.knoe.dev + per-engineer onboarding,
GCS-backed CNPG backups via Workload Identity, the env-contamination
guard, the Junie brief queue, the cnpg-grafana CSRF + memory-request
fixes from today), while main accumulated Junie's parallel knoe-auth
Phase 2 OIDC work (full provider surface: discovery, authorize, token,
userinfo, JWKS, RS256 signing, code exchange, session services).

Key decision: the two branches did COMPETING rebrands off the same
starting point (5ba9b63, 2026-04-27):

  - claude branch (commit b355855, earlier): org.prole.authority.* →
                                              dev.knoe.auth.*
                                              (artifact renamed to
                                              knoe-auth.jar)
  - main (commit 9daa94b, recent): org.prole.authority.* →
                                    dev.knoe.authority.*
                                    (kept "authority" artifact name)

dev.knoe.auth wins: cluster runs from this name, the Maven artifact is
already knoe-auth.jar, and the broader rename is the documented
namespace direction (per ~/.claude/projects/-Users-chrisfu-dev-knoe-db/
memory/MEMORY.md). All of main's recent Phase 2 OIDC content was ported
from authority/src/.../dev/knoe/authority/ into
authority/src/.../dev/knoe/auth/ with package declarations rewritten.

== File-level resolution summary ==

Textual conflicts (4):

  authority/pom.xml
    - Took our artifactId="auth"
    - Took our branch's removal of spring-security-kerberos-client
      (verified: Junie's Phase 2 OIDC code does not import it; the dep
      was already-dead config)

  docs/pipeline-phases.md
    - Took our branch's "Phase 1 not started" status. Main had a
      misplaced "✅ Complete" with a knoe-auth-Phase-1 commit ref
      in the autobuild Phase 1 section — different domain.

  docs/plans/knoe-auth-round-1.md
    - Took our branch's dev.knoe.auth file table (vs main's
      dev.knoe.authority listing). Pure rename mismatch.

  supabase/helm/knoe-supabase/templates/kong/config.yaml
    - Took our branch's onboard route + plain dashboard wiring.
      Main had an oauth2proxy.enabled toggle that put oauth2-proxy as
      a Kong upstream — but the deployed architecture (commit 25f1b2e)
      has oauth2-proxy in FRONT of Kong, not behind. Main's wrapper
      reflected an architecture that was never deployed.
    - Took our branch's removal of basic-auth from dashboard route
      (queue #15 brief still tracks the matching values.yaml /
      kong/deployment.yaml cleanup).

Java tree reconciliation (44 file-pairs):

  20 dual-path source files + 2 dual-path tests
    Body-identical between main's authority/ and our branch's auth/
    after stripping package decls — main's commit 9daa94b was a pure
    rebrand. Took our branch's auth/ version for all 22.

  8 main-only source files (Phase 2 OIDC), ported into auth/:
    web/JwksController.java
    web/OidcAuthorizeController.java
    web/OidcDiscoveryController.java
    web/OidcTokenController.java
    web/OidcUserInfoController.java
    session/OidcCodeService.java
    session/OidcTokenService.java
    session/SessionService.java

  12 main-only test files, ported into auth/:
    HealthControllerTest.java
    enroll/EnrollValueTypesTest.java
    enroll/EnrollmentControllerTest.java
    enroll/TotpServiceTest.java
    kerberos/KadminClientTest.java
    kerberos/KerberosSpnegoResultTest.java
    web/LoginControllerTest.java
    admin/AdminControllerTest.java
    user/PrincipalNormalizerTest.java
    regression/IdentityRegressionTest.java
    session/OidcCodeServiceTest.java
    session/SessionServiceTest.java

  Port mechanics: read main:authority/...<file> via git show, then sed
  rewrite `package dev.knoe.authority` → `package dev.knoe.auth` and
  `import dev.knoe.authority` → `import dev.knoe.auth`. Body content
  unchanged.

  authority/src/main/java/dev/knoe/authority/ — DELETED (duplicate)
  authority/src/test/java/dev/knoe/authority/  — DELETED (duplicate)

== Verification ==

- grep -rln '<<<<<<<' across .java/.md/.yaml/.yml/.sh/.xml/.tpl: clean
- find authority/src -path '*/dev/knoe/authority*': empty (subtree gone)
- grep 'package dev.knoe.authority' across repo: clean
- bash -n install.sh deploy.sh etc/preflight_kubecontext.sh: clean
- git ls-files -u | wc -l: 0 unmerged paths
- helm lint supabase/helm/knoe-supabase: pre-existing failure on
  studioIngress.enabled undefined in values.yaml (introduced by Junie
  on main; unrelated to this merge — flagging as follow-up).

== Followups (carried into TODO ranked queue or noted here) ==

  - helm lint failure: studioIngress block in values.yaml is missing
    enable flag; templates/studio/{ingress,oauth2proxy-deployment,
    oauth2proxy-service}.yaml all reference studioIngress.enabled with
    no default. Pre-existing on main; not introduced by this merge.
  - The five Junie briefs filed on this branch are now reachable from
    main at docs/plans/junie/{02,06,07,13,15}-*.md. Junie can pick them
    up in any order.
  - knoe-auth Phase 2 OIDC source (now at dev.knoe.auth.*) is not yet
    deployed to the cluster. Deployment is its own task.
  - The branch claude/crazy-bose-fec256 stays in place (worktree at
    .claude/worktrees/crazy-bose-fec256 may have ongoing context for
    Claude Code sessions). Safe to delete once next session starts
    cleanly from main.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-01 16:39:10 -07:00

123 lines
5.0 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# AGENTS.md — AI coding agent guide for knoe-db / Knoe
## What this repo is
**Knoe** is an infrastructure stack for deploying a Supabase-style internal developer platform (PostgreSQL, object storage, secrets, auth, observability) across K3d (local), K3s (on-prem), and GKE (cloud) environments. The Python "Knoe" installer (`knoe/cli.py`) drives all cluster setup via milestones.
---
## Architecture overview
### Dual-cluster GKE layout (prod)
Two GKE clusters in `us-west3`:
| Cluster | Context | Purpose |
|---|---|---|
| `knoe-dev-0` | `gke_plenary-truck-485623-p7_us-west3_knoe-dev-0` | App cluster — Garage, Registry, OpenBao, Kong, GitLab, monitoring |
| `knoe-dev-cnpg-0` | `gke_plenary-truck-485623-p7_us-west3_knoe-dev-cnpg-0` | DB cluster — CNPG/PostgreSQL only |
**Critical:** SSD quota (300 GB) is fully consumed by CNPG — all non-CNPG PVCs must use `standard` storage class (HDD), not `standard-rwo`/`premium-rwo`. **Garage runs ONLY on `knoe-dev-0`** (removed from DB cluster on 2026-04-29).
### Deployment environments / modes
| `cluster_env` | `KNOE_MODE` | Target |
|---|---|---|
| `dev` | `k3d` | Local K3d cluster |
| `service` | `k3s` | On-prem K3s cluster |
| `prod` | `k8s` | GKE (or other cloud) |
`_deployment_mode_from_env()` in `knoe/core/env.py` converts env strings to mode strings.
### Config file mapping
`knoe/knoe_conf.py` maps environments to config files under `conf/`:
- `dev` → `k3d.cfg`
- `service` → `k3s.cfg`
- `prod` → `gke.cfg`
Config is layered: env-specific file overrides base. `KNOE_CONF` env var or `conf/service/` subdirs point to the active config.
---
## Key source locations
| Path | Purpose |
|---|---|
| `knoe/core/env.py` | Core config/env helpers, secret encryption, kubeconfig resolution |
| `knoe/core/actions.py` | All installer actions and unattended workflow helpers (~8k lines) |
| `knoe/milestone.py` | `Milestone` ABC — all install steps implement this; `_get_script_env()` builds the env for subprocesses |
| `knoe/knoe_conf.py` | Config path resolution and layered loading |
| `knoe/core/milestones.py` | Concrete milestone definitions |
| `conf/gke.cfg` | Production GKE config (must have correct `app_cluster_kubecontext` / `db_cluster_kubecontext`) |
| `conf/service/prod.cfg` | Unattended deploy config for `./deploy.sh` |
| `etc/` | Shell init scripts (`init_*.sh`) called by milestones |
| `k8s/` | Kubernetes manifests by service |
| `scripts/reset_clusters.sh` | Full cluster teardown + recreate |
---
## Developer workflows
### Install dependencies
```bash
make requirements # pip install -r requirements.txt
```
### Run tests
```bash
make test # runs pyconv (black check) then pytest with coverage
# or directly:
PYTHONPATH=. pytest tests/
```
### Build the knoe CLI binary
```bash
make build # PyInstaller one-file binary → dist/knoe
```
### Interactive installer (ncurses)
```bash
make knoe # launches ./install.sh
# or via the unified launcher:
./knoe.sh install
```
### Unattended deploy
```bash
./deploy.sh # reads conf/service/prod.cfg
```
### Code style
```bash
black . # formatter (black --check . is enforced in CI)
```
---
## Secret handling
Secrets in `knoe.cfg` are AES-GCM encrypted at rest using `${KNOE_SECRET:v1:...}` tokens. On macOS, the key is in Keychain (`knoe-installer` service); on Linux, at `~/.knoe/secrets/knoe.key`. OpenBao references use `${OPENBAO:kv/knoe/<ns>/<leaf>#<key>}`. Never store plaintext passwords in config files.
---
## Milestone pattern
All installer steps subclass `Milestone` (`knoe/milestone.py`). They must:
- Be UI-agnostic (no tkinter/ncurses imports)
- Use `_run_cmd()` for subprocesses (handles env injection)
- Use `_get_script_env(state)` to build env dicts for shell scripts — this is where `KUBECONTEXT`, `DB_CLUSTER_KUBECONTEXT`, `KNOE_CONF`, etc. are set
Missing `init_cluster.app_cluster_kubecontext` in config causes Garage to deploy to the wrong cluster.
---
## CNPG / backup specifics
- CNPG backups go to **GCS**: `gs://knoe-0-backups/` (single bucket; `knoe-db/base/` and `knoe-db/wals/` prefixes). `gs://knoe-0-wal/` exists but is unused.
- CNPG operator: **v1.29.0** (upgraded 2026-04-29 to expose `spec.serviceAccountName`)
- plugin-barman-cloud: v0.12.0
- GCP SA: `cnpg-backup@plenary-truck-485623-p7.iam.gserviceaccount.com` (`storage.objectAdmin` + `storage.legacyBucketReader` on the bucket)
- K8s SA: cluster pods run as **`cnpg-backup-sa`** in `knoe-db-0`, set via `cluster.spec.serviceAccountName: cnpg-backup-sa`. The SA has the `iam.gke.io/gcp-service-account` annotation. RoleBindings `knoe-db` and `knoe-db-barman-cloud` include `cnpg-backup-sa` as a subject so the pod has the same RBAC the auto-generated SA would have had.
- ObjectStore manifest: `k8s/knoe/knoe-db-barman-objectstore-gcs.yaml` — includes `googleCredentials.gkeEnvironment: true`
- Setup: `etc/init_cnpg_gke.sh` and `etc/init_cnpg_backup.sh`
> **Performance note:** Pod `memory: 512Mi` makes `barman-cloud-backup` runs 30–90 min for the 9 GB DB. Bump pending in [`docs/TODO.md`](docs/TODO.md).
Reference in New Issue View Git Blame Copy Permalink