prole/docs/plans/junie/ekosystem-uuid-cnpg-wire.md

Junie brief: ekosystem UUID system — CNPG wiring and follow-on work

Status: Schema files written, init_prole_app.sql wired. CNPG manifests and downstream integrations pending.
Priority: Medium — work through tasks in order; Tasks 1–3 are independent, Task 4 depends on Task 1.

---

Background

A stable routable base36 UUID system has been built for ekosystem (tenant-based federated DB cluster) objects. Two schema files land in knoe-db/schema/:

File	Contents
schema/ekosystem.sql	Core: ekosystem_id(), ekosystem_decode(), ekosystem_tenant(), register_tenant(), project_prefix(), knoe.tenants, knoe.ekosystem_seq
schema/ekosystem_objects.sql	Objects: ekosystem_object(), ekosystem_kind(), object_kinds, cross_grants, grant_object(), revoke_object(), can_access(), my_grants()

Naming conventions in use:

p_{16-char-uuid}            — project namespace / schema root
u_{16-char-uuid}            — service account / PG role
e_{16-char-uuid}            — embedding store
t_{16-char-uuid}_{name}     — data table     (prefix 19 chars, PG budget: 44 for name)
x_{16-char-uuid}_{name}     — index

init_prole_app.sql already wires these for direct psql -f execution. The CNPG production path (which uses inline postInitSQL YAML arrays) needs separate wiring — see Task 1.

Tenant model:

• knoe-db = tenant_id 0, hardcoded root authority
• All other ekosystems (including prole) receive a UUID minted by the root and an assigned integer tenant_id
• Run SELECT * FROM knoe.register_tenant('prole') once on first deploy and record the output

---

Task 1 — Wire schema into all three CNPG manifests via ConfigMap

Why: CNPG postInitSQL is a YAML string array — it cannot reference external files. The functions use PL/pgSQL with dollar-quoting, which is unreadable and fragile when inlined in YAML. The correct CNPG mechanism is a Kubernetes ConfigMap referenced via postInitApplicationSQLRefs.

Files to update:

• k8s/knoe/knoe-db.yaml (k3s production)
• deploy/gcp/gke/knoe-db.yaml (GKE)
• deploy/opentofu/k3s/manifests/knoe/knoe-db.yaml (OpenTofu k3s)

Step 1a — Create the ConfigMap manifest

Create k8s/knoe/knoe-ekosystem-sql.yaml:

apiVersion: v1
kind: ConfigMap
metadata:
  name: knoe-ekosystem-sql
  namespace: knoe-db
data:
  ekosystem.sql: |
    <contents of schema/ekosystem.sql verbatim>    
  ekosystem_objects.sql: |
    <contents of schema/ekosystem_objects.sql verbatim>    

Use a Makefile target or a generation script to keep the ConfigMap content in sync with the source SQL files rather than hand-editing. Suggested target: make k8s/knoe/knoe-ekosystem-sql.yaml.

Step 1b — Reference from CNPG cluster manifests

In each CNPG manifest, append to the bootstrap.initdb section after the existing postInitSQL block. The ekosystem SQL must run after extensions and role setup, so order matters:

      postInitApplicationSQLRefs:
        configMapRefs:
          - name: knoe-ekosystem-sql
            key: ekosystem.sql
          - name: knoe-ekosystem-sql
            key: ekosystem_objects.sql

Note: postInitApplicationSQLRefs was added in CNPG v1.20. Verify against the running operator version before applying:

kubectl -n cnpg-system get deployment cnpg-controller-manager \
  -o jsonpath='{.spec.template.spec.containers[0].image}'

Step 1c — Apply and verify

kubectl apply -f k8s/knoe/knoe-ekosystem-sql.yaml

# Verify functions exist after next cluster bootstrap or via psql:
psql "host=pg.prole.org ..." -c "\df knoe.ekosystem_*"
psql "host=pg.prole.org ..." -c "SELECT * FROM knoe.ekosystem_decode(knoe.ekosystem_id(0,0));"

---

Task 2 — Register prole as tenant 1

Why: The knoe.tenants table currently only has the root row (tenant_id=0, knoe-db). Prole needs to be registered so its project UUIDs embed the correct tenant_id=1 in their bits.

Steps:

1. Connect to the production DB via pg.prole.org
2. Run: SELECT * FROM knoe.register_tenant('prole');
3. Record the returned uuid — this is prole's stable external identifier:
   • DNS: db.{uuid}.prole.org
   • Shown in service-account prefixes: u_{uuid}
4. Save the UUID in conf/ or a new docs/ekosystem-tenants.md for reference
5. When generating prole project IDs, use tenant_id=1: SELECT knoe.ekosystem_id(1, 0);

---

Task 3 — Python counterpart utility

Why: ClickHouse and SQLite nodes cannot call PostgreSQL functions. Application-layer code (Python) needs to mint ekosystem IDs without a PG connection.

Location: src/knoe/ekosystem.py (or lib/ekosystem.py depending on project layout)

Spec:

EPOCH_MS   = 1767225600000  # 2026-01-01 00:00:00 UTC
ALPHABET   = '0123456789abcdefghijklmnopqrstuvwxyz'

# Bit layout: [49:ts_ms | 12:tenant | 10:shard | 11:seq]
# Must produce identical output to knoe.ekosystem_id() in PostgreSQL.

class EkosystemID:
    def __init__(self, tenant_id: int, shard_id: int): ...
    def generate(self) -> str: ...           # thread-safe, per-ms atomic seq
    def project_prefix(self) -> str: ...     # 'p_{uuid}_'
    def object_name(self, kind: str) -> str: # '{prefix}_{uuid}'

def decode(uid: str) -> dict: ...            # tenant_id, shard_id, ts_ms, seq, issued_at
def can_access(uid: str, kind: str, tenant_id: int, grants: list) -> bool: ...

The per-ms atomic counter (seq) should use threading.Lock() — same Snowflake pattern as the SQL version. No global database locking.

Include a round-trip test: assert decode(EkosystemID(1, 0).generate())['tenant_id'] == 1

---

Task 4 — Align knoe.user with ekosystem user UUIDs

Why: knoe.user (in the CNPG manifests) uses SERIAL integer IDs. Users created via the ekosystem system get u_{uuid} PG role names. These two identity systems need a join column so that a Kerberos/LDAP principal can be resolved to both their knoe.user.id (for existing app code) and their ekosystem UUID (for routing and LDAP group membership in cross-grants).

Steps:

1. Add ekosystem_uuid text UNIQUE column to knoe.user:

   ALTER TABLE knoe.user ADD COLUMN IF NOT EXISTS ekosystem_uuid text UNIQUE;
   

2. When a new user is provisioned, call knoe.ekosystem_id(tenant_id, shard_id) to mint their UUID and write it to ekosystem_uuid
3. Add a migration entry in the CNPG manifests' postInitSQL for the new column (or apply directly to the live DB and track in docs/TODO.md)

---

Task 5 — LDAP/Samba AD: wire cross_grants to AD group membership

Why: knoe.cross_grants is the policy source of truth for cross-ekosystem sharing. The Samba AD domain controller (myrddin.prole.org) enforces access at the auth layer. These need to be kept in sync.

Spec (design only — implementation requires knoe-auth work to be further along):

• A periodic reconciler reads knoe.my_grants(tenant_id) for each registered tenant
• For each active received grant, it ensures the grantee's AD user/group is a member of the AD group corresponding to the grantor's project
• AD group naming convention: ekosystem-{object_uuid} (stays under 64-char SAM-Account-Name limit at 10+16=26 chars)
• Revoking a grant removes the AD group membership

Prerequisite: pg_oauth / pg-knoe-auth OIDC path must be further along so the reconciler can authenticate to PG using a service Kerberos principal.

---

Acceptance criteria (all tasks)

-- Task 1: functions exist in production
SELECT routine_name FROM information_schema.routines
WHERE routine_schema = 'knoe' AND routine_name LIKE 'ekosystem%';

-- Task 2: prole registered
SELECT * FROM knoe.tenants;
-- expect: (0, NULL, 'knoe-db'), (1, '{16-char-uuid}', 'prole')

-- Task 3: Python output matches SQL output for same inputs
-- (unit test in test suite)

-- Task 4: knoe.user has ekosystem_uuid column
\d knoe.user

-- Task 5: design doc only for now