mirror of
https://github.com/dredx/prole.git
synced 2026-09-24 19:54:32 +00:00
Bringing the long-running session-feature branch back into main in one deliberate sweep. The branch carried the cluster work that's been live for weeks (cross-cluster CNPG metrics, Grafana w/ Google OAuth, supabase oauth2-proxy, cluster recovery, pg.0.knoe.dev + per-engineer onboarding, GCS-backed CNPG backups via Workload Identity, the env-contamination guard, the Junie brief queue, the cnpg-grafana CSRF + memory-request fixes from today), while main accumulated Junie's parallel knoe-auth Phase 2 OIDC work (full provider surface: discovery, authorize, token, userinfo, JWKS, RS256 signing, code exchange, session services). Key decision: the two branches did COMPETING rebrands off the same starting point (5ba9b63, 2026-04-27): - claude branch (commit b355855, earlier): org.prole.authority.* → dev.knoe.auth.* (artifact renamed to knoe-auth.jar) - main (commit9daa94b, recent): org.prole.authority.* → dev.knoe.authority.* (kept "authority" artifact name) dev.knoe.auth wins: cluster runs from this name, the Maven artifact is already knoe-auth.jar, and the broader rename is the documented namespace direction (per ~/.claude/projects/-Users-chrisfu-dev-knoe-db/ memory/MEMORY.md). All of main's recent Phase 2 OIDC content was ported from authority/src/.../dev/knoe/authority/ into authority/src/.../dev/knoe/auth/ with package declarations rewritten. == File-level resolution summary == Textual conflicts (4): authority/pom.xml - Took our artifactId="auth" - Took our branch's removal of spring-security-kerberos-client (verified: Junie's Phase 2 OIDC code does not import it; the dep was already-dead config) docs/pipeline-phases.md - Took our branch's "Phase 1 not started" status. Main had a misplaced "✅ Complete" with a knoe-auth-Phase-1 commit ref in the autobuild Phase 1 section — different domain. docs/plans/knoe-auth-round-1.md - Took our branch's dev.knoe.auth file table (vs main's dev.knoe.authority listing). Pure rename mismatch. supabase/helm/knoe-supabase/templates/kong/config.yaml - Took our branch's onboard route + plain dashboard wiring. Main had an oauth2proxy.enabled toggle that put oauth2-proxy as a Kong upstream — but the deployed architecture (commit 25f1b2e) has oauth2-proxy in FRONT of Kong, not behind. Main's wrapper reflected an architecture that was never deployed. - Took our branch's removal of basic-auth from dashboard route (queue #15 brief still tracks the matching values.yaml / kong/deployment.yaml cleanup). Java tree reconciliation (44 file-pairs): 20 dual-path source files + 2 dual-path tests Body-identical between main's authority/ and our branch's auth/ after stripping package decls — main's commit9daa94bwas a pure rebrand. Took our branch's auth/ version for all 22. 8 main-only source files (Phase 2 OIDC), ported into auth/: web/JwksController.java web/OidcAuthorizeController.java web/OidcDiscoveryController.java web/OidcTokenController.java web/OidcUserInfoController.java session/OidcCodeService.java session/OidcTokenService.java session/SessionService.java 12 main-only test files, ported into auth/: HealthControllerTest.java enroll/EnrollValueTypesTest.java enroll/EnrollmentControllerTest.java enroll/TotpServiceTest.java kerberos/KadminClientTest.java kerberos/KerberosSpnegoResultTest.java web/LoginControllerTest.java admin/AdminControllerTest.java user/PrincipalNormalizerTest.java regression/IdentityRegressionTest.java session/OidcCodeServiceTest.java session/SessionServiceTest.java Port mechanics: read main:authority/...<file> via git show, then sed rewrite `package dev.knoe.authority` → `package dev.knoe.auth` and `import dev.knoe.authority` → `import dev.knoe.auth`. Body content unchanged. authority/src/main/java/dev/knoe/authority/ — DELETED (duplicate) authority/src/test/java/dev/knoe/authority/ — DELETED (duplicate) == Verification == - grep -rln '<<<<<<<' across .java/.md/.yaml/.yml/.sh/.xml/.tpl: clean - find authority/src -path '*/dev/knoe/authority*': empty (subtree gone) - grep 'package dev.knoe.authority' across repo: clean - bash -n install.sh deploy.sh etc/preflight_kubecontext.sh: clean - git ls-files -u | wc -l: 0 unmerged paths - helm lint supabase/helm/knoe-supabase: pre-existing failure on studioIngress.enabled undefined in values.yaml (introduced by Junie on main; unrelated to this merge — flagging as follow-up). == Followups (carried into TODO ranked queue or noted here) == - helm lint failure: studioIngress block in values.yaml is missing enable flag; templates/studio/{ingress,oauth2proxy-deployment, oauth2proxy-service}.yaml all reference studioIngress.enabled with no default. Pre-existing on main; not introduced by this merge. - The five Junie briefs filed on this branch are now reachable from main at docs/plans/junie/{02,06,07,13,15}-*.md. Junie can pick them up in any order. - knoe-auth Phase 2 OIDC source (now at dev.knoe.auth.*) is not yet deployed to the cluster. Deployment is its own task. - The branch claude/crazy-bose-fec256 stays in place (worktree at .claude/worktrees/crazy-bose-fec256 may have ongoing context for Claude Code sessions). Safe to delete once next session starts cleanly from main. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
226 lines
16 KiB
Markdown
226 lines
16 KiB
Markdown
# Resync `prole` as a Customer Deploy of `knoe-db`
|
|
|
|
**Status:** **Dormant.** The plan was written when `~/dev/prole` and `~/dev/knoe-db` were two separate working trees diverging on GitHub vs the knoe Gitea. As of 2026-04, the prole→knoe rebrand has merged into `knoe-db/main` directly, there is no separate `~/dev/prole` working tree under active development, and both `origin` and `knoe` remotes in this repo point at upstream `git@git-ssh.knoe.dev:knoe.dev/knoe-db.git`. The customer-branch workflow described below is the **target shape** for any future per-customer separation; it has not been executed and has no current customer driving it. Sections below are preserved as the design reference if the workflow is reactivated.
|
|
**Owner:** chrisfu
|
|
**Audience:** Jr/mid engineer joining the project. No prior knowledge of the prole↔knoe-db split assumed.
|
|
|
|
---
|
|
|
|
## 1. Context
|
|
|
|
This working tree (`~/dev/prole`) and `~/dev/knoe-db` started life as the same project. Over time `knoe-db` has become the canonical upstream — the "Knoe" infrastructure platform, hosted at `git@git-ssh.knoe.dev:knoe.dev/knoe-db.git` — while `prole` has continued as a separate fork on GitHub with its own diverging history.
|
|
|
|
The strategic decision the team has made is: **stop forking, start branching.** `knoe-db` is the platform's source of truth. Customer deployments — currently `prole.org`, eventually others — live as **branches in `knoe-db`**, not as separate repositories. A customer branch carries only the customer-specific divergence (config files, branding, on-prem manifests, kubeconfig handling). Platform changes always land on `knoe-db/main` and customer branches rebase or merge from main on a regular cadence.
|
|
|
|
This plan describes how to bring `~/dev/prole` into that shape.
|
|
|
|
### Why a branch and not a fork
|
|
|
|
- **One pull-request review surface for platform changes.** A change that affects all deployments goes through MR review on `knoe-db/main` once. In a fork model the same change has to be re-reviewed (or, more often, silently re-implemented) per fork.
|
|
- **Customer-specific divergence is observable.** A `git diff main..prole` is the complete answer to "what's special about the prole.org deploy?" In a fork model that question requires a manual cross-repo diff that quickly goes stale.
|
|
- **Resync is mechanical.** `git rebase main` or `git merge main` updates the deploy branch. In a fork model resync is a recurring tax that engineers underinvest in until the forks have drifted past easy reconciliation — which is roughly where prole is today.
|
|
- **Customer-specific platform changes get attention.** If a customer needs a platform-level change to support their deploy, it lands on `main` (and is therefore available to all customers) instead of being hidden in a fork that never makes it back.
|
|
|
|
### What's already done
|
|
|
|
- prole has `knoe-db` configured as a git remote named `knoe`:
|
|
```
|
|
knoe git@git-ssh.knoe.dev:knoe.dev/knoe-db.git (fetch)
|
|
knoe git@git-ssh.knoe.dev:knoe.dev/knoe-db.git (push)
|
|
origin git@github.com:dredx/prole.git (fetch)
|
|
origin git@github.com:dredx/prole.git (push)
|
|
```
|
|
So the network plumbing for `git fetch knoe` is in place. `origin` is the legacy GitHub fork.
|
|
|
|
- `knoe-db` already has `docs/plans/` with the architectural references for `knoe-auth Round 1` and the four-mode installer — see [`knoe-db/docs/plans/`](../../../knoe-db/docs/plans/) (path relative to your `~/dev/` root).
|
|
|
|
### What's not yet done
|
|
|
|
- Most of `prole/main` has not seen `knoe-db/main`'s recent work (the prole→knoe rebrand, the welcome-mode-selector Phase 0, the fully implemented `authority/` module, the 1Password CLI cutover, etc.).
|
|
- Prole still tracks `infrastructure/` (the prole.org-specific Ansible tree) which `knoe-db` excised in commit `5a769f2`.
|
|
- Prole still tracks the old `prole-*` directory names that knoe-db has renamed (`prole-app/`, `prole-tools-app/`, `prole.sh`, `prole.spec`, `prole_requirements.txt`).
|
|
- Prole's working tree carries credentials and runtime artifacts that should never have been near git in the first place — `postgres-password.txt`, `prole-k3s.kubeconfig*`, `secrets/`, `ssh-keys/`. None are committed (verified with `git ls-files`), but they exist in the working copy and are a data-leak risk.
|
|
|
|
---
|
|
|
|
## 2. Target shape
|
|
|
|
When this plan is done:
|
|
|
|
- The `prole` working directory tracks a branch in `knoe-db` — name to be confirmed, proposed `customer/prole.org` or simply `prole`.
|
|
- `git remote` looks like:
|
|
```
|
|
origin git@git-ssh.knoe.dev:knoe.dev/knoe-db.git
|
|
```
|
|
The legacy GitHub `origin` is either retired entirely or kept as `archive` for historical reference.
|
|
- `git diff origin/main` (i.e. `knoe-db/main`) shows **only** prole-specific divergence:
|
|
- Customer-branded config (`conf/prole.cfg`, host names, realm `PROLE.LOCAL`, etc.)
|
|
- On-prem manifests scoped to `prole.org` infrastructure (`deploy/opentofu/k3s/manifests/prole/*`)
|
|
- Customer-specific docs (`docs/plans/customer-deploy-resync.md` — this file — and any future prole-only plans)
|
|
- All tracked platform code (`authority/`, `knoe/`, `etc/init_*.sh`, `deploy/gcp/gke/*`, the test pipeline) is **byte-identical** to `knoe-db/main`. Changes to those files happen on `knoe-db/main` first and reach prole via merge/rebase.
|
|
- Working-tree credentials and runtime files are removed, gitignored, and replaced with the 1Password-driven equivalents already used in `knoe-db`.
|
|
|
|
### What "customer-specific" actually means here
|
|
|
|
It's a short list, and the discipline is to keep it short:
|
|
|
|
| Type | Example | Where it lives on the branch |
|
|
| --- | --- | --- |
|
|
| Customer config | `conf/prole.cfg` (cluster contexts, realm, hostnames) | `conf/` |
|
|
| Customer realm | `PROLE.LOCAL` Kerberos manifests | `deploy/opentofu/k3s/manifests/prole/prole-kdc-*.yaml` |
|
|
| Customer cluster glue | k3s `prole.org` host inventory, anything host-pinned to `merlin/myrddin/pi.prole.org` | `deploy/opentofu/k3s/manifests/prole/` |
|
|
| Customer docs | `docs/plans/customer-deploy-resync.md`, any future prole-only specs | `docs/plans/` |
|
|
|
|
Anything that doesn't fit one of those four lines is probably platform code that should land on `main`.
|
|
|
|
---
|
|
|
|
## 3. Migration plan
|
|
|
|
The migration is in four phases. Each phase is reviewable on its own; do not collapse them.
|
|
|
|
> **All git work in this plan should run in Claude Code, not in Cowork.** Cowork's bash sandbox has no network access and cannot reach `git-ssh.knoe.dev`. Open a Code session in `~/dev/prole` and drive the commands from there.
|
|
|
|
### Phase 1 — Snapshot and inventory
|
|
|
|
The goal of this phase is to produce a complete account of what's currently special about prole, *before* anyone starts deleting things. Output is a file checked into the working tree (`docs/plans/resync-inventory.md`, net-new) listing every file that differs from `knoe-db/main`, classified as one of:
|
|
|
|
- **PLATFORM-DRIFT** — file should match `knoe-db/main`. Discrepancy is just stale prole. Action: take knoe-db's version.
|
|
- **CUSTOMER-SPECIFIC** — file is genuinely prole-only. Action: keep, plan to land on the customer branch.
|
|
- **WORKING-TREE-NOISE** — credentials, build artifacts, kubeconfigs. Action: gitignore upstream if not already; remove from working tree; never commit.
|
|
- **AMBIGUOUS** — needs human review. Action: discuss before acting.
|
|
|
|
Concrete commands (run in Code in `~/dev/prole`):
|
|
|
|
```bash
|
|
# fetch knoe-db
|
|
git fetch knoe
|
|
|
|
# list all files that exist in prole but not in knoe-db/main
|
|
git diff --name-only --diff-filter=D knoe/main..HEAD > /tmp/prole-only.txt
|
|
|
|
# list all files that exist in both but differ
|
|
git diff --name-only knoe/main..HEAD | grep -v '^docs/plans/customer-deploy-resync.md$' > /tmp/prole-modified.txt
|
|
|
|
# list all files that exist in knoe-db/main but not in prole
|
|
git diff --name-only --diff-filter=A knoe/main..HEAD > /tmp/knoe-db-new.txt
|
|
```
|
|
|
|
Walk through each list and classify. Expected hot spots based on the current state:
|
|
|
|
- `infrastructure/` — entirely PLATFORM-DRIFT (excised upstream by commit `5a769f2`).
|
|
- `prole-app/`, `prole-auth/`, `prole-mssql-db/`, `prole-net/`, `prole-tools-app/`, `prole.sh`, `prole.spec`, `prole_requirements.txt` — PLATFORM-DRIFT (renamed upstream during the prole→knoe rebrand).
|
|
- `authority/src/main/java/org/prole/authority/HealthController.java` etc. — PLATFORM-DRIFT (stale Spring Boot skeleton; upstream has the full module).
|
|
- `deploy/opentofu/k3s/manifests/prole/*` — likely CUSTOMER-SPECIFIC (the `prole.org` realm and host pinning belongs on the customer branch).
|
|
- `conf/prole.cfg`, `conf/prole.silent.all.cfg` — CUSTOMER-SPECIFIC.
|
|
- `postgres-password.txt`, `prole-k3s.kubeconfig*`, `secrets/`, `ssh-keys/` — WORKING-TREE-NOISE. These should not be tracked anywhere.
|
|
|
|
Output of phase 1 is the inventory file plus a short summary commit on a fresh branch (proposed name `resync/phase-1-inventory`).
|
|
|
|
### Phase 2 — Bring `prole/main` to `knoe-db/main`
|
|
|
|
> ⚠️ **Read this whole phase before running any of it.** This is where the divergence collapses; recovery from a botched merge is much harder than waiting an hour to do it right.
|
|
|
|
Approach: do **not** try to merge upstream into prole's diverged main. Instead:
|
|
|
|
1. Create a backup tag at the current state: `git tag prole-pre-resync-$(date +%Y%m%d)`. Push it to `origin` (the GitHub fork) for safety.
|
|
2. Create a fresh customer-deploy branch off `knoe-db/main`:
|
|
```bash
|
|
git checkout -b customer/prole.org knoe/main
|
|
```
|
|
3. Cherry-pick **only** the CUSTOMER-SPECIFIC commits/files from the inventory. The cleanest way is a one-pass apply:
|
|
```bash
|
|
# for each file in the customer-specific list, copy from old prole main
|
|
git checkout prole-pre-resync-<date> -- <path>
|
|
git add <path>
|
|
```
|
|
Commit in coherent groups (e.g. one commit per logical scope: "config", "k3s manifests", "customer docs").
|
|
4. Push the new branch to the `knoe` remote:
|
|
```bash
|
|
git push knoe customer/prole.org
|
|
```
|
|
5. Open an MR in `knoe-db` for the customer branch (it doesn't merge to main — it's a long-lived branch — but the MR makes it visible and reviewable).
|
|
|
|
After this phase, `customer/prole.org` is byte-identical to `main` except for the (small) customer-specific delta.
|
|
|
|
### Phase 3 — Retire the GitHub fork and clean the working tree
|
|
|
|
1. Update the local working tree's `origin` remote to point at `knoe`:
|
|
```bash
|
|
git remote rename origin archive
|
|
git remote rename knoe origin
|
|
git fetch origin
|
|
```
|
|
Or, if you want to delete the GitHub fork outright, drop the `archive` remote. Either is fine; the GitHub copy is no longer authoritative.
|
|
2. Switch the working tree to the new branch:
|
|
```bash
|
|
git checkout customer/prole.org
|
|
```
|
|
3. Delete working-tree noise (the WORKING-TREE-NOISE bucket from phase 1). For each file:
|
|
- Confirm it's not tracked: `git ls-files --error-unmatch <path>`.
|
|
- Confirm it's covered by `.gitignore` (or add it).
|
|
- Delete it: `rm -rf <path>`.
|
|
- Special case: `postgres-password.txt`. Treat the contents as compromised because it was sitting in a working tree synced to who-knows-where. **Rotate the password through the standard process**, do not just delete the file.
|
|
4. Replicate `knoe-db`'s 1Password CLI workflow: `etc/` already has the 1Password integration after `knoe-db` commit `b421f49`. Set up the 1Password CLI and verify `op signin` works before relying on the new flow.
|
|
|
|
### Phase 4 — Establish the resync cadence
|
|
|
|
Going forward, prole resyncs from `knoe-db/main` on a deliberate cadence (proposal: weekly, or after any commit on `main` that touches a file the customer branch carries a divergent copy of).
|
|
|
|
Mechanism — pick one:
|
|
|
|
- **Rebase** `customer/prole.org` onto `knoe-db/main`. Linear history, clean diff against main, but rewrites the customer-branch SHAs on every resync. Choose this if no other tooling has hardcoded references to customer-branch commits.
|
|
- **Merge** `knoe-db/main` into `customer/prole.org`. Preserves history, no SHA rewriting, but the `git log --oneline` gets messy and `git diff main..customer/prole.org` includes merge artifacts.
|
|
|
|
Recommendation: **rebase**. Customer divergence is small enough to keep clean, and the simpler diff is worth more than the merge-commit history.
|
|
|
|
Set up CI on `knoe-db` to run the customer-deploy smoke test against the rebased branch nightly. Failure of that smoke test is an early signal that platform changes have broken the customer deploy and need a customer-side fix.
|
|
|
|
---
|
|
|
|
## 4. Out of scope
|
|
|
|
These are real items but they are **not** part of the resync. Each becomes its own plan when its time comes.
|
|
|
|
- Migrating the *running* prole.org cluster to the new branch's deployment shape. Resync is a code-level operation; the live cluster is a separate migration governed by `etc/init_*.sh` and the installer.
|
|
- Bringing prole's existing Supabase work back into knoe-db/main. Recent prole commits include Supabase fixes (`fix(supabase): ...`) that aren't in upstream. Some of that may be platform code that should land on `main`; some may be prole-specific. Sort that out as a follow-up MR.
|
|
- The active "knoe-db Init:0/1 on myrddin" debug. That's a runtime-state problem on the prole.org cluster, unrelated to the source-code resync. Track in its own ticket.
|
|
- Onboarding additional customer deploys. Once one customer branch is healthy, the second is mostly mechanical, but not part of this plan's scope.
|
|
|
|
---
|
|
|
|
## 5. Verification
|
|
|
|
The resync is done when all of these are true.
|
|
|
|
1. `git remote -v` in `~/dev/prole` shows `origin` pointing at `knoe-db`. The legacy GitHub remote is either gone or named `archive`.
|
|
2. `git status` is clean on branch `customer/prole.org`.
|
|
3. `git diff knoe/main` (or `git diff origin/main` post-rename) shows **only** files in the four categories listed in §2 ("What 'customer-specific' actually means"). No `infrastructure/`, no `prole-app/`, no stale Java skeletons.
|
|
4. `git ls-files` does not list `postgres-password.txt`, `prole-k3s.kubeconfig*`, or any file under `secrets/` or `ssh-keys/`.
|
|
5. The Postgres password that was previously sitting in `postgres-password.txt` has been rotated.
|
|
6. `./install.sh` or `./knoe.sh install` runs end-to-end on the customer branch in `min` and `k3d` modes (the cheap modes to verify locally).
|
|
7. `customer/prole.org` is pushed to `knoe-db` and visible in the upstream repo's branch list.
|
|
8. `docs/plans/customer-deploy-resync.md` (this file) and `docs/plans/resync-inventory.md` (created in phase 1) are committed on the customer branch.
|
|
|
|
---
|
|
|
|
## 6. Risks and call-outs
|
|
|
|
- **Don't run any of this with the prole.org cluster in a degraded state if the cluster recovery depends on a script that the resync deletes.** The current `knoe-db` Init:0/1 debug is a live ops problem; finish that first, or be very careful that the scripts you're about to delete from prole are not the ones the cluster needs to come back.
|
|
- **The Postgres password file is a real incident.** A credential file in a working tree synced via Dropbox / iCloud / network home directory has a non-trivial blast radius. Treat as compromised, rotate, do not just delete.
|
|
- **The GitHub `origin` may have copies of secrets in its history.** Before retiring the fork, audit its commit history for committed credentials (`git log -p | grep -iE 'password|secret|token|key'` is a starting point). If anything is in there, rotate before retiring. GitHub's "delete repository" does not retroactively un-disclose anything that was scraped while the repo was up.
|
|
|
|
---
|
|
|
|
## 7. Glossary
|
|
|
|
**Upstream / downstream** — Upstream is the canonical project (`knoe-db`); downstream is a deployment of it (`prole.org` customer branch). Changes flow from upstream to downstream, never the other way without an MR back.
|
|
|
|
**Customer deploy** — A specific deployment of the knoe.dev platform for one organization. Carries customer-specific config (hostnames, realm, branding) on top of the platform's main branch. Lives as a long-lived branch in the platform repo, not a fork.
|
|
|
|
**Long-lived branch** — A branch that's not intended to merge back to main. It's continuously rebased onto main as platform work lands, but its own divergence stays as a divergence forever.
|
|
|
|
**Resync** — The act of bringing a customer branch up to a newer `main`. A periodic, deliberate operation; not a continuous one.
|
|
|
|
**Phase 0** — The most recent platform-side rebrand and test-pipeline foundation work (commit `0052a4d` in `knoe-db`). Context here only because it's what `customer/prole.org` is rebasing onto.
|