prole/docs/plans/knoe-auth-round-1.md
chrisfu 00f0ebec07 Merge claude/crazy-bose-fec256 into main
Bringing the long-running session-feature branch back into main in one
deliberate sweep. The branch carried the cluster work that's been live for
weeks (cross-cluster CNPG metrics, Grafana w/ Google OAuth, supabase
oauth2-proxy, cluster recovery, pg.0.knoe.dev + per-engineer onboarding,
GCS-backed CNPG backups via Workload Identity, the env-contamination
guard, the Junie brief queue, the cnpg-grafana CSRF + memory-request
fixes from today), while main accumulated Junie's parallel knoe-auth
Phase 2 OIDC work (full provider surface: discovery, authorize, token,
userinfo, JWKS, RS256 signing, code exchange, session services).

Key decision: the two branches did COMPETING rebrands off the same
starting point (5ba9b63, 2026-04-27):

  - claude branch (commit b355855, earlier): org.prole.authority.* →
                                              dev.knoe.auth.*
                                              (artifact renamed to
                                              knoe-auth.jar)
  - main (commit 9daa94b, recent): org.prole.authority.* →
                                    dev.knoe.authority.*
                                    (kept "authority" artifact name)

dev.knoe.auth wins: cluster runs from this name, the Maven artifact is
already knoe-auth.jar, and the broader rename is the documented
namespace direction (per ~/.claude/projects/-Users-chrisfu-dev-knoe-db/
memory/MEMORY.md). All of main's recent Phase 2 OIDC content was ported
from authority/src/.../dev/knoe/authority/ into
authority/src/.../dev/knoe/auth/ with package declarations rewritten.

== File-level resolution summary ==

Textual conflicts (4):

  authority/pom.xml
    - Took our artifactId="auth"
    - Took our branch's removal of spring-security-kerberos-client
      (verified: Junie's Phase 2 OIDC code does not import it; the dep
      was already-dead config)

  docs/pipeline-phases.md
    - Took our branch's "Phase 1 not started" status. Main had a
      misplaced " Complete" with a knoe-auth-Phase-1 commit ref
      in the autobuild Phase 1 section — different domain.

  docs/plans/knoe-auth-round-1.md
    - Took our branch's dev.knoe.auth file table (vs main's
      dev.knoe.authority listing). Pure rename mismatch.

  supabase/helm/knoe-supabase/templates/kong/config.yaml
    - Took our branch's onboard route + plain dashboard wiring.
      Main had an oauth2proxy.enabled toggle that put oauth2-proxy as
      a Kong upstream — but the deployed architecture (commit 25f1b2e)
      has oauth2-proxy in FRONT of Kong, not behind. Main's wrapper
      reflected an architecture that was never deployed.
    - Took our branch's removal of basic-auth from dashboard route
      (queue #15 brief still tracks the matching values.yaml /
      kong/deployment.yaml cleanup).

Java tree reconciliation (44 file-pairs):

  20 dual-path source files + 2 dual-path tests
    Body-identical between main's authority/ and our branch's auth/
    after stripping package decls — main's commit 9daa94b was a pure
    rebrand. Took our branch's auth/ version for all 22.

  8 main-only source files (Phase 2 OIDC), ported into auth/:
    web/JwksController.java
    web/OidcAuthorizeController.java
    web/OidcDiscoveryController.java
    web/OidcTokenController.java
    web/OidcUserInfoController.java
    session/OidcCodeService.java
    session/OidcTokenService.java
    session/SessionService.java

  12 main-only test files, ported into auth/:
    HealthControllerTest.java
    enroll/EnrollValueTypesTest.java
    enroll/EnrollmentControllerTest.java
    enroll/TotpServiceTest.java
    kerberos/KadminClientTest.java
    kerberos/KerberosSpnegoResultTest.java
    web/LoginControllerTest.java
    admin/AdminControllerTest.java
    user/PrincipalNormalizerTest.java
    regression/IdentityRegressionTest.java
    session/OidcCodeServiceTest.java
    session/SessionServiceTest.java

  Port mechanics: read main:authority/...<file> via git show, then sed
  rewrite `package dev.knoe.authority` → `package dev.knoe.auth` and
  `import dev.knoe.authority` → `import dev.knoe.auth`. Body content
  unchanged.

  authority/src/main/java/dev/knoe/authority/ — DELETED (duplicate)
  authority/src/test/java/dev/knoe/authority/  — DELETED (duplicate)

== Verification ==

- grep -rln '<<<<<<<' across .java/.md/.yaml/.yml/.sh/.xml/.tpl: clean
- find authority/src -path '*/dev/knoe/authority*': empty (subtree gone)
- grep 'package dev.knoe.authority' across repo: clean
- bash -n install.sh deploy.sh etc/preflight_kubecontext.sh: clean
- git ls-files -u | wc -l: 0 unmerged paths
- helm lint supabase/helm/knoe-supabase: pre-existing failure on
  studioIngress.enabled undefined in values.yaml (introduced by Junie
  on main; unrelated to this merge — flagging as follow-up).

== Followups (carried into TODO ranked queue or noted here) ==

  - helm lint failure: studioIngress block in values.yaml is missing
    enable flag; templates/studio/{ingress,oauth2proxy-deployment,
    oauth2proxy-service}.yaml all reference studioIngress.enabled with
    no default. Pre-existing on main; not introduced by this merge.
  - The five Junie briefs filed on this branch are now reachable from
    main at docs/plans/junie/{02,06,07,13,15}-*.md. Junie can pick them
    up in any order.
  - knoe-auth Phase 2 OIDC source (now at dev.knoe.auth.*) is not yet
    deployed to the cluster. Deployment is its own task.
  - The branch claude/crazy-bose-fec256 stays in place (worktree at
    .claude/worktrees/crazy-bose-fec256 may have ongoing context for
    Claude Code sessions). Safe to delete once next session starts
    cleanly from main.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-01 16:39:10 -07:00

31 KiB
Raw Blame History

knoe-auth Round 1 — Durable Kerberos Identity + Contributor Onboarding

Status: Implemented and operational. Architectural reference. Owner: chrisfu Audience: Jr/mid engineer onboarding to knoe.dev. No prior Kerberos or OIDC experience assumed.


1. Context

knoe-auth is the identity system for the knoe.dev platform — the thing that decides who you are, what you can access, and how new contributors come on board. It is a long-term, multi-round initiative. This document is the architectural reference for Round 1, which is shipped.

The chicken-and-egg problem

To build a sophisticated identity provider safely, the team needs to be able to authenticate themselves against something trustworthy in the meantime. We needed a working auth store before we could safely build the better one.

Round 1 leans on a tool that has been doing this job for thirty-five years: MIT Kerberos. It is unfashionable but well-understood, cryptographically sound, and we already had it running in our k3s homelab cluster. We extended it to GKE, wrapped a small enrollment portal around it, and use that to onboard contributing engineers while Round 2 (a full OIDC provider) is being built.

Why Kerberos

Kerberos principals are durable, DNS-like identifiers. chrisfu@KNOE.DEV is a stable cryptographic identity that survives any change to our web stack, our database, or our cloud provider. Once a principal exists in the KDC, it can issue tickets that any kerberized service trusts — Spring Boot via SPNEGO, Postgres via gss auth, SSH, NFS — without each of those services needing its own user table.

That's the asset Round 1 builds on.

Two independent Google Workspaces — do not conflate

Workspace Role
knoe.dev Internal Google Workspace for the platform. Has zero pre-knowledge of any contributor's home org.
prole.org Workspace of the first contributing engineer's organization. Independently operated.

knoe.dev does not trust prole.org as a domain. prole.org is just one engineer's email provider, no different from gmail.com or any other workspace a future contributor might use. Trust between knoe.dev and a new contributor is bootstrapped by the invite, not by the contributor's home Google domain.

Trust model — the most important paragraph in this document

When a new engineer enrolls, the trust sequence is:

  1. An admin sends an invite to a specific email address or phone number. That contact channel — and only that channel — is the trust anchor. The admin's choice of who to invite is the policy.
  2. The engineer proves they control that contact by entering a one-time password (OTP) delivered to it. Until the OTP verifies, no further steps are possible.
  3. Only after the OTP gate is the engineer offered a Google sign-in to corroborate their identity. The Google sign-in is welcomed in after trust is already established by the invite — it is not the source of trust.
  4. TOTP (the rotating six-digit code from Google Authenticator / Authy) is set up as the ongoing 2FA credential.
  5. Only then is a Kerberos principal minted, a knoe.user row inserted, and downstream provisioning jobs (GitLab account, Gitea account) queued.

This means knoe.dev never needs to pre-configure trust with any external workspace. The OAuth2 app does not restrict by Google hd (hosted domain) — any verified Google account works. The contributor's home domain is recorded for audit (knoe.identity.provider_hd) but never used to gate access.

If you remember nothing else: the invite OTP is the trust anchor. Google is corroboration. TOTP is the ongoing factor.


2. How it's wired — file map

The actual files that implement Round 1. Verify with git ls-files before assuming any of the below has rotted.

Java application — authority/

The Spring Boot service that implements the enrollment flow, the admin API, and SPNEGO-protected endpoints. Two-pom Maven build (root pom.xml for knoe-db controller, authority/pom.xml for knoe-auth); the application package is dev.knoe.auth and the Maven coordinate is dev.knoe:auth.

File Responsibility
authority/src/main/java/dev/knoe/auth/KnoeAuthApplication.java @SpringBootApplication entry point.
authority/src/main/java/dev/knoe/auth/HealthController.java /health endpoint.
authority/src/main/java/dev/knoe/auth/web/LoginController.java Form-login + SPNEGO challenge for browsers without a ticket.
authority/src/main/java/dev/knoe/auth/web/VerifyController.java Token-verify endpoint for downstream services.
authority/src/main/java/dev/knoe/auth/session/SessionTokenService.java Issues HMAC-SHA256 JWT cookies after successful auth.
authority/src/main/java/dev/knoe/auth/session/SessionUser.java Authenticated principal carried in the security context.
authority/src/main/java/dev/knoe/auth/user/PrincipalNormalizer.java Strips realm/instance from a Kerberos principal (alice/admin@KNOE.DEValice).
authority/src/main/java/dev/knoe/auth/kerberos/KerberosSpnegoService.java SPNEGO challenge/response handling.
authority/src/main/java/dev/knoe/auth/kerberos/KerberosPasswordService.java Password-style auth fallback for browsers that can't do SPNEGO.
authority/src/main/java/dev/knoe/auth/kerberos/KadminClient.java Shells out to kadmin.local (in the KDC sidecar) to addprinc and cpw. Sanitizes input.
authority/src/main/java/dev/knoe/auth/enroll/EnrollmentController.java Web endpoints: GET /auth/enroll, POST /auth/enroll/verify-otp, POST /auth/enroll/identity/start, GET /auth/enroll/google-callback, GET /auth/enroll/totp, POST /auth/enroll/totp/verify, POST /auth/enroll/complete.
authority/src/main/java/dev/knoe/auth/enroll/InviteService.java CRUD + validation against knoe.invitation. OTP hashing (bcrypt) and rate limiting (3 attempts).
authority/src/main/java/dev/knoe/auth/enroll/GoogleOAuthService.java Exchange OAuth2 code → ID token, validate email_verified, return a GoogleIdentity record. No hd allowlist.
authority/src/main/java/dev/knoe/auth/enroll/TotpService.java Generate TOTP secret, produce otpauth:// URI, verify codes.
authority/src/main/java/dev/knoe/auth/enroll/UserProvisioningService.java Transactional orchestrator: inserts user/identity/totp rows, calls KadminClient, queues provisioning jobs.
authority/src/main/java/dev/knoe/auth/admin/AdminController.java POST /auth/admin/invites, GET /auth/admin/users, POST /auth/admin/grants. SPNEGO + admin-role gated.
authority/src/main/java/dev/knoe/auth/admin/KnobjectService.java CRUD on knoe.knobject and knoe.access_grant; enqueues provisioning_job rows.
authority/src/main/java/dev/knoe/auth/provisioning/ProvisioningWorker.java @Scheduled poller for knoe.provisioning_job WHERE status = 'pending'. Dispatches to GitLab/Gitea/CNPG.
authority/src/main/java/dev/knoe/auth/config/AuthProperties.java Typed binding for knoe.auth.* keys.
authority/src/main/java/dev/knoe/auth/config/KerberosProperties.java Typed binding for knoe.auth.kerberos.* keys.

Tests for the above live under authority/src/test/java/dev/knoe/auth/ — notably web/VerifyControllerTest.java and session/SessionTokenServiceTest.java.

Kubernetes manifests — GKE

File Purpose
deploy/gcp/gke/knoe-kdc-configmap.yaml krb5.conf + kdc.conf for the KNOE.DEV realm.
deploy/gcp/gke/knoe-kdc-secrets.yaml Master key + admin password references. Production values come from OpenBao.
deploy/gcp/gke/knoe-auth-deployment.yaml KDC sidecar + Spring Boot pod. Service principal HTTP/auth.knoe.dev@KNOE.DEV.
deploy/gcp/gke/knoe-auth-google-oidc-secret.example.yaml Templated Secret with GOOGLE_CLIENT_ID / GOOGLE_CLIENT_SECRET placeholders. Real values are not committed.
deploy/gcp/gke/workload-identity.yaml KSA↔GSA bindings for any GCP-backed secret access.

Kubernetes manifests — k3s (homelab and customer deploys)

File Purpose
deploy/opentofu/k3s/manifests/knoe/prole-kdc-configmap.yaml The PROLE.LOCAL realm KDC for the homelab cluster. The pattern the GKE configmap was modeled on.
deploy/opentofu/k3s/manifests/knoe/prole-kdc-secrets.example.yaml Templated secrets for the same.
deploy/opentofu/k3s/manifests/knoe/knoe-auth-deployment.yaml KDC + Spring Boot for the homelab.
deploy/opentofu/k3s/manifests/knoe/knoe-auth-kerberos-configmap.yaml krb5.conf for the auth pod's Kerberos client.

Bootstrap scripts — etc/

File Purpose
etc/init_kdc.sh Provisions the in-cluster KDC. Idempotent. Read this end-to-end before writing anything that interacts with the KDC.
etc/init_knoe_users.sh Creates knoe.user, knoe.user_role, and (per Round 1) the additional auth tables; seeds initial principals via kadmin.local.
etc/init_kerberos.sh Cluster-wide krb5.conf wiring for kerberized services (Postgres, etc.).

3. Architecture

The invite-to-enrolled flow at a glance

Invite URL
https://auth.knoe.dev/enroll?token=<uuid>
   │
   ├─ Step 1: Enter OTP (delivered to invite email/phone)
   │           ─ Trust anchor. Without this, no further steps.
   │
   ├─ Step 2: Pick a username, link with Google (any account, any hd)
   │           ─ Corroboration. Records provider_sub + provider_hd for audit.
   │
   ├─ Step 3: Scan QR with authenticator app, verify TOTP code
   │           ─ Sets up the ongoing 2FA factor.
   │
   └─ Step 4: System provisions:
              ├─ Kerberos principal: <username>@KNOE.DEV
              ├─ knoe.user row + knoe.identity link to Google subject
              ├─ knoe.totp_credential row (encrypted secret)
              └─ Async queue: GitLab account, Gitea account, …

Cluster topology

Round 1 lives in the GKE app cluster (knoe-dev-0) in the knoe-system namespace. The database stays where it already is — the dedicated CNPG cluster knoe-dev-cnpg-0. See CLAUDE.md for the cluster layout and storage-quota rules.

knoe-dev-0 / knoe-system namespace:
  ┌─ knoe-auth (Spring Boot) ─────────────────────────────────────────┐
  │  /health                                                           │
  │  /auth/login, /auth/spnego, /auth/verify                           │
  │  /auth/enroll/*  (invite → OTP → Google → TOTP → provision)        │
  │  /auth/admin/*   (create invites, manage knobjects)                │
  └────────────────────────────────────────────────────────────────────┘
           │ kadmin.local calls (KDC is a sidecar in the same pod)
           ▼
  ┌─ knoe-kdc (MIT Kerberos, KNOE.DEV realm) ─────────────────────────┐
  │  Container pattern from prole-kdc-configmap.yaml                   │
  │  Realm: KNOE.DEV                                                   │
  │  Cross-realm trust with PROLE.LOCAL: deferred to Round 2           │
  └────────────────────────────────────────────────────────────────────┘
           │
           ▼
  ┌─ CNPG / knoe-db (in knoe-dev-cnpg-0) ─────────────────────────────────┐
  │  knoe.user             (base table)                                │
  │  knoe.user_role        (base table)                                │
  │  knoe.invitation       (Round 1)                                   │
  │  knoe.identity         (Round 1 — Google sub → knoe user)          │
  │  knoe.totp_credential  (Round 1 — encrypted TOTP secret + backups) │
  │  knoe.knobject         (Round 1 — platform resources)              │
  │  knoe.access_grant     (Round 1 — user → knobject grants)          │
  │  knoe.provisioning_job (Round 1 — async outbox)                    │
  └────────────────────────────────────────────────────────────────────┘

The KDC runs as a sidecar in the same pod as the Spring Boot app. They share the pod network namespace, so kadmin.local calls reach the KDC over loopback — no Kubernetes Service required between them. This is the same pattern the k3s deployment uses.


4. Schema

The knoe.* schema lives in CNPG (knoe-dev-cnpg-0 namespace knoe-db-0). Base tables (knoe.user, knoe.user_role) are created by etc/init_knoe_users.sh lines 482510. Round 1 added the six tables below; they are created by the same script later in its run.

-- ────────────────────────────────────────────────────────────────────
-- knoe.invitation — admin creates one of these per invited engineer.
-- The (contact, otp_hash) pair IS the trust anchor for that engineer.
-- ────────────────────────────────────────────────────────────────────
CREATE TABLE IF NOT EXISTS knoe.invitation (
  id              UUID PRIMARY KEY DEFAULT gen_random_uuid(),
  token           TEXT NOT NULL UNIQUE,         -- URL token (long, random)
  contact         TEXT NOT NULL,                -- email or phone the invite was sent to
  contact_type    TEXT NOT NULL DEFAULT 'email',-- 'email' | 'sms'
  name_hint       TEXT,                         -- optional display-name hint from admin
  otp_hash        TEXT NOT NULL,                -- bcrypt of the 6-digit OTP
  otp_expires_at  TIMESTAMPTZ NOT NULL,         -- short TTL (10 min)
  otp_attempts    INT NOT NULL DEFAULT 0,       -- max 3 before invalidation
  otp_verified_at TIMESTAMPTZ,                  -- set when OTP passes — gate for steps 2-4
  created_by      TEXT NOT NULL,                -- admin username
  created_at      TIMESTAMPTZ NOT NULL DEFAULT now(),
  expires_at      TIMESTAMPTZ NOT NULL,         -- invite URL TTL (72h)
  used_at         TIMESTAMPTZ,                  -- set at step-4 completion
  used_by         TEXT                          -- knoe username after use
);

-- ────────────────────────────────────────────────────────────────────
-- knoe.identity — external identity corroborations.
-- Round 1 only writes Google rows here; future providers reuse the table.
-- ────────────────────────────────────────────────────────────────────
CREATE TABLE IF NOT EXISTS knoe.identity (
  id             SERIAL PRIMARY KEY,
  user_id        INT NOT NULL REFERENCES knoe.user(id) ON DELETE CASCADE,
  provider       TEXT NOT NULL,                 -- 'google'
  provider_sub   TEXT NOT NULL,                 -- Google subject ID (stable per user)
  provider_email TEXT,
  provider_hd    TEXT,                          -- 'prole.org' | 'gmail.com' | NULL — audit only
  verified_at    TIMESTAMPTZ NOT NULL,
  UNIQUE(provider, provider_sub)
);

-- ────────────────────────────────────────────────────────────────────
-- knoe.totp_credential — the rotating 2FA factor for ongoing logins.
-- ────────────────────────────────────────────────────────────────────
CREATE TABLE IF NOT EXISTS knoe.totp_credential (
  user_id      INT PRIMARY KEY REFERENCES knoe.user(id) ON DELETE CASCADE,
  secret       TEXT NOT NULL,                   -- AES-GCM encrypted, key in OpenBao
  verified_at  TIMESTAMPTZ,                     -- NULL until first successful verification
  backup_codes TEXT[],                          -- bcrypt-hashed one-time recovery codes
  created_at   TIMESTAMPTZ NOT NULL DEFAULT now()
);

-- ────────────────────────────────────────────────────────────────────
-- knoe.knobject — platform-managed resources (a "knobbed object",
-- something an admin can hand to a user).
-- ────────────────────────────────────────────────────────────────────
CREATE TABLE IF NOT EXISTS knoe.knobject (
  id          SERIAL PRIMARY KEY,
  type        TEXT NOT NULL,                    -- 'gitea_repo' | 'gitlab_project' | 'cnpg_role' | 'openbao_policy'
  name        TEXT NOT NULL,
  platform_id TEXT,                             -- external identifier on target platform
  metadata    JSONB,
  created_at  TIMESTAMPTZ NOT NULL DEFAULT now(),
  UNIQUE(type, name)
);

-- ────────────────────────────────────────────────────────────────────
-- knoe.access_grant — user ← knobject with role.
-- ────────────────────────────────────────────────────────────────────
CREATE TABLE IF NOT EXISTS knoe.access_grant (
  id           SERIAL PRIMARY KEY,
  user_id      INT NOT NULL REFERENCES knoe.user(id),
  knobject_id  INT NOT NULL REFERENCES knoe.knobject(id),
  role         TEXT NOT NULL,                   -- 'owner' | 'developer' | 'viewer'
  granted_by   TEXT NOT NULL,
  granted_at   TIMESTAMPTZ NOT NULL DEFAULT now(),
  revoked_at   TIMESTAMPTZ,
  UNIQUE(user_id, knobject_id)
);

-- ────────────────────────────────────────────────────────────────────
-- knoe.provisioning_job — async outbox.
-- A worker bean inside knoe-auth polls this table.
-- ────────────────────────────────────────────────────────────────────
CREATE TABLE IF NOT EXISTS knoe.provisioning_job (
  id         SERIAL PRIMARY KEY,
  user_id    INT NOT NULL REFERENCES knoe.user(id),
  job_type   TEXT NOT NULL,                     -- 'create_gitlab_user' | 'create_gitea_user' | 'grant_cnpg_role'
  status     TEXT NOT NULL DEFAULT 'pending',   -- 'pending' | 'running' | 'done' | 'failed'
  payload    JSONB,
  result     JSONB,
  created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
  updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
);

5. Configuration

authority/.../application.properties — relevant keys

knoe.auth.google.client-id=${GOOGLE_CLIENT_ID}
knoe.auth.google.client-secret=${GOOGLE_CLIENT_SECRET}
knoe.auth.google.redirect-uri=https://auth.knoe.dev/auth/enroll/google-callback

# No allowed-domains list. Trust is established by invite OTP, not by the
# developer's home Google domain. provider_hd is recorded in knoe.identity
# for audit, never used for access control.

knoe.auth.enroll.invite-ttl-hours=72
knoe.auth.enroll.otp-ttl-minutes=10
knoe.auth.enroll.otp-max-attempts=3
knoe.auth.enroll.totp-issuer=knoe.dev

knoe.auth.provisioning.poll-interval-ms=10000

pom.xml — relevant dependencies

<!-- TOTP -->
<dependency>
  <groupId>dev.samstevens.totp</groupId>
  <artifactId>totp-spring-boot-starter</artifactId>
  <version>1.7.1</version>
</dependency>

<!-- Google OAuth2 client -->
<dependency>
  <groupId>com.google.api-client</groupId>
  <artifactId>google-api-client</artifactId>
  <version>2.4.0</version>
</dependency>

Google OAuth2 app

Configured by hand in the GCP console under project plenary-truck-485623-p7.

  • Authorized redirect URIs:
    • https://auth.knoe.dev/auth/enroll/google-callback (Round 1 enrollment)
    • https://git.knoe.dev/... (Gitea OIDC, future round)
    • https://git.prole.org/... (GitLab OIDC, future round)
  • No hd= restriction at the OAuth app level. Deliberate.
  • Client ID and secret stored in a Kubernetes Secret modeled on deploy/gcp/gke/knoe-auth-google-oidc-secret.example.yaml.

6. Step-by-step enrollment flow

This is what the engineer being onboarded actually experiences, what the server actually does, and where the trust transitions live.

Step 0 — Admin creates the invite

POST /auth/admin/invites
Body: { contact: "chrisfu@prole.org", contact_type: "email", name_hint: "Chris Fu" }

Server:
  generate invite_token (UUID v4)
  generate otp (6-digit numeric, 10-min TTL)
  hash otp with bcrypt → otp_hash
  insert into knoe.invitation
  send email to chrisfu@prole.org:
    Subject: "You've been invited to knoe.dev"
    Body:    invite URL  +  "Your verification code: 847291"

The OTP is delivered through the same channel as the invite URL. Both arrive in the engineer's inbox. The OTP is not sent to a side channel — its purpose is to prove possession of that inbox.

Step 1 — Engineer proves contact ownership (the trust gate)

GET /auth/enroll?token=<uuid>
  → InviteService validates token is not expired/used
  → renders landing.html with the OTP entry form

POST /auth/enroll/verify-otp  { otp: "847291" }
  → InviteService compares bcrypt(otp) with otp_hash, checks expiry
  → on success: set otp_verified_at=now(), advance session to step 2
  → on failure: increment otp_attempts; if >= 3, invalidate the invite and return 403

After this step, the session is gated. The remaining steps are unreachable without a verified OTP. This is where the trust transition happens.

GET  /auth/enroll/identity
  → renders identity.html — username field + display-name field +
    "Sign in with Google" button

POST /auth/enroll/identity/start  { username: "chrisfu", display_name: "Chris Fu" }
  → store proposed username + display name in session
  → redirect to Google OAuth2 authorize URL
    (state=<session_id>, nonce=<random>, NO hd parameter)

GET  /auth/enroll/google-callback?code=<code>&state=<session_id>
  → GoogleOAuthService exchanges code for ID token
  → validates id_token.email_verified == true (REQUIRED)
  → records provider_sub, provider_email, provider_hd
    (provider_hd is whatever Google reports — prole.org, gmail.com, etc.)
  → stores GoogleIdentity in session, advances to step 3

This step is corroboration, not authorization. The engineer's home Google workspace is not a trust source. We accept any verified Google account and record which domain it came from for audit purposes.

Step 3 — Engineer sets up TOTP

GET  /auth/enroll/totp
  → TotpService.generateSecret() — 160-bit base32 secret
  → store the encrypted secret in the session (NOT yet in the DB)
  → render totp-setup.html with:
      • a QR code encoding otpauth://totp/knoe.dev:<username>?secret=...&issuer=knoe.dev
      • the 16-character manual key for users with no QR scanner
      • "Open Google Authenticator / Authy and scan this code"

POST /auth/enroll/totp/verify  { code: "123456" }
  → TotpService.verify(sessionSecret, code) — validates within ±1 30-second window
  → on success: advance to step 4
  → on failure: re-render with the same secret (don't rotate yet)

Step 4 — System provisions

POST /auth/enroll/complete
  → UserProvisioningService.provision(session) runs in a single transaction:
       1. INSERT INTO knoe.user (username, realm='KNOE.DEV', email, display_name)
       2. INSERT INTO knoe.identity (provider='google', sub, email, hd)
       3. INSERT INTO knoe.totp_credential (AES-encrypted secret, verified_at=now())
       4. KadminClient.addPrincipal("<username>@KNOE.DEV")
       5. UPDATE knoe.invitation SET used_at=now(), used_by=<username>
       6. INSERT INTO knoe.provisioning_job (job_type='create_gitea_user', payload={...})
       7. INSERT INTO knoe.provisioning_job (job_type='create_gitlab_user', payload={...})
  → render complete.html with the engineer's new username and a "what happens next"
    summary (their dev environment is being set up async, they'll get a follow-up email).

Properties of this flow you can rely on

  • The OTP delivery channel is the identity proof. If the OTP arrives, the engineer owns that inbox.
  • knoe.dev never trusted prole.org. It trusted the admin's choice to send the invite to a prole.org address. Different thing.
  • The Google link captures the engineer's home workspace as audit data, but does not gate access.
  • TOTP becomes the ongoing 2FA factor. The Google sign-in is a one-time corroboration; logins after enrollment use Kerberos + TOTP.
  • knoe.identity.provider_hd records the home domain without pre-judging it.

7. Verification

Run this end-to-end after any change touching the auth/Kerberos surface.

  1. kubectl -n knoe-system get podsknoe-auth and knoe-kdc both Running.
  2. curl https://auth.knoe.dev/health returns 200.
  3. Hit POST /auth/admin/invites from an admin SPNEGO session, receive an invite URL.
  4. Open the enrollment URL in a fresh browser, complete all four steps using a real Google account in a non-knoe.dev workspace (e.g. gmail.com).
  5. psql … -c "SELECT username, realm FROM knoe.user WHERE username = '<test_user>';" returns the row.
  6. kinit <test_user>@KNOE.DEV from a machine that trusts the realm — succeeds.
  7. psql … -c "SELECT job_type, status FROM knoe.provisioning_job WHERE user_id = (SELECT id FROM knoe.user WHERE username = '<test_user>');" shows create_gitea_user and create_gitlab_user rows.
  8. After the polling interval, those rows transition to status = 'done' and the corresponding accounts exist on the platforms.

8. Out of scope for Round 1

Real, named items the team has discussed. They are not in Round 1 — when the team gets to them, each becomes its own plan in this directory.

  • Cross-realm trust between KNOE.DEV and PROLE.LOCAL (so a chrisfu@PROLE.LOCAL ticket can talk to a KNOE.DEV service). Round 2.
  • A full OIDC provider hosted by knoe-auth, replacing the dependence on Google for downstream services. Round 2.
  • SSO into kerberized Postgres roles (gss auth) so knoe.user rows map directly to database principals.
  • A "knobject inspector" admin UI. Right now the admin API is JSON-only.
  • Phone/SMS-based OTP delivery. Round 1 covers email; the contact_type column is already present on knoe.invitation so adding SMS later is additive.
  • Self-service password rotation, recovery flows, and deactivation. Admin-only for Round 1.
  • Round 1.5 — OpenBao transit encryption for secrets at rest. UserProvisioningService currently writes the per-user TOTP secret and any other long-lived per-user secrets to knoe.* tables in the clear (relying only on PG-level encryption-at-rest). Round 1.5 wraps writes/reads with an OpenBao transit-key envelope so the database can't yield plaintext secrets even if it leaks. Tracked by the // TODO Round 1.5 marker in authority/src/main/java/dev/knoe/auth/enroll/UserProvisioningService.java.

9. Glossary

KDC — Key Distribution Center. The Kerberos server. Holds the master key for the realm; issues TGTs (ticket-granting tickets) and service tickets.

Kerberos principal — A named identity in a realm. Format: name@REALM (or service/host@REALM). Example: chrisfu@KNOE.DEV. Long-lived, cryptographic, decoupled from any particular service's user table.

Realm — A Kerberos administrative domain. Independent KDCs each own their own realm. PROLE.LOCAL and KNOE.DEV are two realms; cross-realm trust is configured separately.

Keytab — A file containing one or more principals' long-term keys, used by services to authenticate to the KDC without an interactive password. Spring Boot reads its service principal's keytab at startup.

SPNEGO — Simple and Protected GSSAPI Negotiation Mechanism. The HTTP-layer protocol that lets a browser holding a Kerberos ticket authenticate to a web app over the wire. RFC 4178. Spring Security has built-in support.

kadmin / kadmin.local — The KDC's administration tool. kadmin runs over the network with admin credentials; kadmin.local runs on the KDC host itself, bypassing the network protocol. Round 1 calls kadmin.local from a sidecar container.

TGT (Ticket-Granting Ticket) — The first ticket the KDC issues to a user after they prove their identity. Used to request further service tickets without re-entering credentials.

TOTP — Time-based One-Time Password. RFC 6238. The rotating six-digit code Google Authenticator and Authy show. A shared secret + the current Unix time bucketed into 30-second windows produces the code.

OAuth2 — Authorization framework. Lets a user grant an app limited access to their account at another service. Concerns delegation, not identity.

OIDC (OpenID Connect) — An identity layer on top of OAuth2. Adds the id_token (a signed JWT with claims about the user). When we say "Google sign-in" we mean OIDC over Google's OAuth2.

hd claim — In a Google OIDC id_token, the user's hosted-domain (i.e. their Google Workspace). Optional, present only for Workspace accounts. We record it in knoe.identity.provider_hd for audit but do not gate access on it.

Workload Identity — GKE feature that binds a Kubernetes ServiceAccount to a Google Cloud IAM service account. Lets pods talk to GCP APIs (e.g. KMS for our master key) without long-lived JSON keyfiles.

CNPG — CloudNativePG. The Postgres operator we use for knoe-db. Runs in knoe-dev-cnpg-0. See CLAUDE.md for cluster topology.

knobject — A platform-managed resource that an admin can hand out to a user — a Gitea repo, a Postgres role, an OpenBao policy. The name is a portmanteau of "knoe" + "object". Modeled by knoe.knobject; granted via knoe.access_grant.

Round 1, Round 2, … — Versioning convention for the auth roadmap. Each round is a self-contained, shippable increment. Round 1 stops where the platform can safely onboard contributing engineers; Round 2 introduces the OIDC provider; later rounds tighten the screws.