# AGENTS.md — AI coding agent guide for knoe-db / Knoe

## What this repo is

**Knoe** is an infrastructure stack for deploying a Supabase-style internal developer platform (PostgreSQL, object storage, secrets, auth, observability) across K3d (local), K3s (on-prem), and GKE (cloud) environments. The Python "Knoe" installer (`knoe/cli.py`) drives all cluster setup via milestones.

---

## Architecture overview

### Dual-cluster GKE layout (prod)
Two GKE clusters in `us-west3`:
| Cluster | Context | Purpose |
|---|---|---|
| `knoe-dev-0` | `gke_plenary-truck-485623-p7_us-west3_knoe-dev-0` | App cluster — Garage, Registry, OpenBao, Kong, GitLab, monitoring |
| `knoe-dev-cnpg-0` | `gke_plenary-truck-485623-p7_us-west3_knoe-dev-cnpg-0` | DB cluster — CNPG/PostgreSQL only |

**Critical:** SSD quota (300 GB) is fully consumed by CNPG — all non-CNPG PVCs must use `standard` storage class (HDD), not `standard-rwo`/`premium-rwo`. **Garage runs ONLY on `knoe-dev-0`** (removed from DB cluster on 2026-04-29).

### Deployment environments / modes
| `cluster_env` | `KNOE_MODE` | Target |
|---|---|---|
| `dev` | `k3d` | Local K3d cluster |
| `service` | `k3s` | On-prem K3s cluster |
| `prod` | `k8s` | GKE (or other cloud) |

`_deployment_mode_from_env()` in `knoe/core/env.py` converts env strings to mode strings.

### Config file mapping
`knoe/knoe_conf.py` maps environments to config files under `conf/`:
- `dev` → `k3d.cfg`
- `service` → `k3s.cfg`
- `prod` → `gke.cfg`

Config is layered: env-specific file overrides base. `KNOE_CONF` env var or `conf/service/` subdirs point to the active config.

---

## Key source locations

| Path | Purpose |
|---|---|
| `knoe/core/env.py` | Core config/env helpers, secret encryption, kubeconfig resolution |
| `knoe/core/actions.py` | All installer actions and unattended workflow helpers (~8k lines) |
| `knoe/milestone.py` | `Milestone` ABC — all install steps implement this; `_get_script_env()` builds the env for subprocesses |
| `knoe/knoe_conf.py` | Config path resolution and layered loading |
| `knoe/core/milestones.py` | Concrete milestone definitions |
| `conf/gke.cfg` | Production GKE config (must have correct `app_cluster_kubecontext` / `db_cluster_kubecontext`) |
| `conf/service/prod.cfg` | Unattended deploy config for `./deploy.sh` |
| `etc/` | Shell init scripts (`init_*.sh`) called by milestones |
| `k8s/` | Kubernetes manifests by service |
| `scripts/reset_clusters.sh` | Full cluster teardown + recreate |

---

## Developer workflows

### Install dependencies
```bash
make requirements        # pip install -r requirements.txt
```

### Run tests
```bash
make test                # runs pyconv (black check) then pytest with coverage
# or directly:
PYTHONPATH=. pytest tests/
```

### Build the knoe CLI binary
```bash
make build               # PyInstaller one-file binary → dist/knoe
```

### Interactive installer (ncurses)
```bash
make knoe                # launches ./install.sh
# or via the unified launcher:
./knoe.sh install
```

### Unattended deploy
```bash
./deploy.sh              # reads conf/service/prod.cfg
```

### Code style
```bash
black .                  # formatter (black --check . is enforced in CI)
```

---

## Secret handling

Secrets in `knoe.cfg` are AES-GCM encrypted at rest using `${KNOE_SECRET:v1:...}` tokens. On macOS, the key is in Keychain (`knoe-installer` service); on Linux, at `~/.knoe/secrets/knoe.key`. OpenBao references use `${OPENBAO:kv/knoe/<ns>/<leaf>#<key>}`. Never store plaintext passwords in config files.

---

## Milestone pattern

All installer steps subclass `Milestone` (`knoe/milestone.py`). They must:
- Be UI-agnostic (no tkinter/ncurses imports)
- Use `_run_cmd()` for subprocesses (handles env injection)
- Use `_get_script_env(state)` to build env dicts for shell scripts — this is where `KUBECONTEXT`, `DB_CLUSTER_KUBECONTEXT`, `KNOE_CONF`, etc. are set

Missing `init_cluster.app_cluster_kubecontext` in config causes Garage to deploy to the wrong cluster.

---

## CNPG / backup specifics

- CNPG backups go to **GCS**: `gs://knoe-0-backups/` (single bucket; `knoe-db/base/` and `knoe-db/wals/` prefixes). `gs://knoe-0-wal/` exists but is unused.
- CNPG operator: **v1.29.0** (upgraded 2026-04-29 to expose `spec.serviceAccountName`)
- plugin-barman-cloud: v0.12.0
- GCP SA: `cnpg-backup@plenary-truck-485623-p7.iam.gserviceaccount.com` (`storage.objectAdmin` + `storage.legacyBucketReader` on the bucket)
- K8s SA: cluster pods run as **`cnpg-backup-sa`** in `knoe-db-0`, set via `cluster.spec.serviceAccountName: cnpg-backup-sa`. The SA has the `iam.gke.io/gcp-service-account` annotation. RoleBindings `knoe-db` and `knoe-db-barman-cloud` include `cnpg-backup-sa` as a subject so the pod has the same RBAC the auto-generated SA would have had.
- ObjectStore manifest: `k8s/knoe/knoe-db-barman-objectstore-gcs.yaml` — includes `googleCredentials.gkeEnvironment: true`
- Setup: `etc/init_cnpg_gke.sh` and `etc/init_cnpg_backup.sh`

> **Performance note:** Pod `memory: 512Mi` makes `barman-cloud-backup` runs 30–90 min for the 9 GB DB. Bump pending in [`docs/TODO.md`](docs/TODO.md).