# Knoe
**Knoe makes it practical to run a Supabase-style platform across air-gapped, edge, and cloud environments.**
It packages the core building blocks needed for a modern internal developer platform around PostgreSQL, object storage, secrets, auth, observability, and Kubernetes-native operations — with a bias toward simple deployment, shard-based scale-out, and deterministic ingestion.
---
## Platform badges
---
## What Knoe is
Knoe is an infrastructure stack for running a Supabase-like data and application platform across constrained, sovereign, and cloud environments, including:
- air-gapped networks
- edge and small-cluster deployments (K3s, K3d)
- Google Cloud Platform (GCP) deployments
- sovereign or private data environments
- disconnected labs and field systems
- internal platforms where cloud dependencies are undesirable
The design goal is not to mimic Supabase branding or every managed feature exactly.
The design goal is to provide the **useful substrate** people actually want:
- PostgreSQL as the center of gravity
- object storage
- auth integration
- secret management
- ingress and service exposure
- observability
- reproducible Kubernetes deployment
- application and worker services around the database
- ingestion patterns that are safe, resumable, and idempotent
---
## Core idea
**Knoe can run Supabase-style workloads in an air gap, on the edge, or in the cloud.**
That means:
- PostgreSQL remains the system of record, managed as `knoe-db` within the `knoe-system` namespace
- services are deployed on Kubernetes (K3d locally, K3s on-prem, GKE on GCP)
- auth can come from local identity systems or cloud OIDC where available
- object storage stays inside the environment
- secrets stay inside the environment
- ingestion does not depend on live internet services
- the stack can be deployed in shards for locality, resilience, and operational simplicity
---
## Main components
### Application layer
- **Next.js app** for the user-facing platform
- **knoe-agent worker** for async and background execution
### Data layer
- **knoe-db** — CloudNativePG-managed PostgreSQL cluster in the `knoe-system` namespace
- **PostgreSQL** as the primary relational store
- **append-only fact tables** for durable ingestion and replay-friendly modeling
- **PostgreSQL COPY** for efficient bulk ingest
### Deployed add-ons
- **Supabase** — self-hosted Supabase stack for auth, realtime, storage, and API gateway
- **ArgoCD** — GitOps continuous delivery for the `knoe-system` cluster
- **Gitea** — lightweight self-hosted Git service
- **GitLab** — full-featured self-hosted DevOps platform
### Storage and secrets
- **Garage** for S3-compatible object storage
- **OpenBao** for secret storage and secret distribution
### Identity
- **Dev auth:** Samba AD + IdP
- **Cloud auth:** Google Workspace OIDC
### Observability
- **Vector** for log and event shipping
- **Prometheus / Grafana** for metrics and dashboards
### Platform operations
- **K3d** — local development clusters
- **K3s** — on-prem lightweight Kubernetes
- **GCP / GKE** — cloud-hosted cluster tier
- **Helm**
- **ArgoCD**
- **Ansible**
- **OpenTofu**
---
## Ingestion model
Knoe is designed around deterministic, replayable ingestion.
Key patterns:
- **SQLite delta cartridges** as portable input units
- **append-only fact tables** for auditability and recovery
- **PostgreSQL COPY** for high-throughput loading
- **queue-driven workers** using `SKIP LOCKED`
- **idempotent ingestion** so retries are safe
- **shard-based deployment** so data movement stays deliberate and bounded
This makes the platform well suited to environments where data may arrive in batches, be transferred physically, or need careful replay and provenance.
---
## Architecture at a glance
```mermaid
flowchart LR
subgraph Clients
U["Users / Operators"]
end
subgraph knoe-system ["knoe-system (Kubernetes)"]
A["Next.js App"]
W["knoe-agent worker"]
SB["Supabase\n(auth · realtime · storage · API)"]
ARGO["ArgoCD\n(GitOps)"]
GITEA["Gitea\n(Git)"]
GITLAB["GitLab\n(DevOps)"]
PG["knoe-db\n(CloudNativePG / PostgreSQL)"]
OBJ["Garage\n(Object Store)"]
SEC["OpenBao\n(Secrets)"]
OBS["Vector / Prometheus / Grafana\n(Observability)"]
ING["Traefik / Kong\n(Ingress)"]
end
subgraph Infra ["Infrastructure"]
K3D["K3d\n(local dev)"]
K3S["K3s\n(on-prem)"]
GCP["GCP / GKE\n(cloud)"]
end
subgraph Identity
AUTH["Samba AD / Google Workspace OIDC"]
end
U --> ING
ING --> A
ING --> SB
ING --> ARGO
ING --> GITEA
ING --> GITLAB
A --> PG
A --> OBJ
A --> SEC
A --> AUTH
W --> PG
W --> OBJ
W --> SEC
SB --> PG
SB --> OBJ
SB --> AUTH
ARGO --> A
ARGO --> W
ARGO --> SB
ARGO --> PG
ARGO --> OBJ
ARGO --> SEC
GITEA --> ARGO
GITLAB --> ARGO
K3D & K3S & GCP --> |hosts| knoe-system
OBS -.-> PG
OBS -.-> A
OBS -.-> W
```
---
## Deployment model
Knoe favors simple, understandable deployment over excessive platform ceremony.
Typical characteristics:
- Kubernetes-native deployment across K3d (local), K3s (on-prem), and GCP (cloud)
- lightweight K3s/K3d-friendly footprint
- HA where it matters
- shard-oriented layout instead of one giant control surface
- GitOps-driven operations via ArgoCD, backed by Gitea or GitLab
- support for small clusters, including Pi-based or edge environments
---
## Use cases
Knoe is a good fit for teams that need:
- a PostgreSQL-centered application stack inside a disconnected or sovereign environment
- a self-hosted substrate for Supabase-style internal platforms
- ingestion from offline or intermittently connected sources
- reproducible deployment on small Kubernetes clusters (K3s/K3d) or GCP
- strong control over secrets, storage, and identity boundaries
- a full GitOps workflow with ArgoCD, Gitea, and GitLab
---
## Project principles
- **Air-gap first**
- **Postgres first**
- **Simple over ornate**
- **Deterministic ingestion**
- **Kubernetes-native**
- **Shard-friendly**
- **Operationally boring where possible**
---
## Repository scope
This repository contains the infrastructure and platform materials used to stand up Knoe components, including Kubernetes deployment, database operations (`knoe-db` in `knoe-system`), storage, auth, observability, and supporting services.
As the project evolves, this README should stay focused on:
- what Knoe is
- why it exists
- the major building blocks
- the deployment model
- the operational philosophy
Detailed setup docs, cluster procedures, and host-specific notes should live in dedicated documents under `docs/`, `ops/`, `k8s/`, or role-specific subdirectories rather than expanding this file indefinitely.
---
## Status
Knoe is an actively evolving platform stack aimed at practical self-hosted, edge, and cloud operation.
Expect the architecture to continue being refined toward:
- cleaner bootstrapping
- better shard isolation
- smoother rejoin/reset behavior for cluster nodes
- clearer service boundaries
- improved onboarding and operations documentation
---
## License
Add the project license here.