prole/docs/knoe-db-documentation-mcp-architecture.md

Knoe-DB Documentation MCP Architecture

Design: Postgres-Core, Next.js Edge

---

1. Objective

Implement a Documentation MCP Server integrated into Prol.app (Next.js) where:

• Postgres (knoe-db) is the only authoritative core
• Next.js provides the MCP interface and UI
• Vector search is optional and derived
• Ingestion is idempotent and Git-versioned
• All responses are citation-grounded and reproducible

There is no Python core and no secondary business-logic layer.
The database owns truth, provenance, and policy.

---

2. High-Level Architecture

Git Repo (docs branch)
        ↓
CI → doc-manifest.json (git_sha, files, hashes)
        ↓
Ingestion Worker (Node k8s Job)
        ↓
Postgres (knoe-db)  ← authoritative core
        ↓
Next.js (Prol.app)
   ├── UI (/docs, /search)
   └── MCP Server (/api/mcp)

Optional vector indexing:

Postgres → embedding worker → pgvector (same DB)

---

3. Core = Postgres Schema

Schema name: doc

Postgres is the authoritative knowledge store.

3.1 doc.source

Tracks document origin and versioning.

Column Type Notes

---

source_id uuid pk
repo text git repository git_sha text commit hash path text file path ingested_at timestamptz
content_hash text integrity check

---

3.2 doc.document

Canonical document metadata.

Column Type Notes

---

doc_id uuid pk
source_id uuid fk references doc.source title text
uri text unique doc://doc/{doc_id} lifecycle text stable, draft, deprecated confidentiality text internal, restricted content_text text full markdown content updated_at timestamptz

---

3.3 doc.chunk

Chunked content for search and embeddings.

Column Type Notes

---

chunk_id uuid pk
doc_id uuid fk
ordinal int chunk order text text
token_count int

---

3.4 doc.embedding (Optional)

Requires pgvector.

Column Type Notes

---

chunk_id uuid pk
embedding vector(1536)
model text
indexed_at timestamptz

---

3.5 doc.link

Semantic relationships between documents.

Column Type

---

from_doc uuid to_doc uuid relation text (applies_to, supersedes, references)

---

4. Policy Enforcement

Default rule:

Only stable documents are searchable unless explicitly overridden.

Enforcement options:

• SQL WHERE clauses in MCP queries (initial phase)
• Row Level Security (future phase)

Confidentiality gating:

• MCP layer passes user_role
• Queries filter by confidentiality <= role_level

Every answer must include:

• doc_id
• uri
• git_sha

The database guarantees provenance.

---

5. Ingestion Pipeline

5.1 Trigger

Git push to docs branch triggers CI.

5.2 CI Output

doc-manifest.json

{
  "repo": "knoe-db",
  "git_sha": "abc123",
  "files": [
    { "path": "runbooks/kerberos.md", "hash": "..." }
  ]
}

5.3 Ingestion Worker (Node.js, Kubernetes Job)

Process:

1. Read manifest
2. For each file:
   • Compute content hash
   • Upsert doc.source
   • Upsert doc.document
   • Chunk content → insert doc.chunk
3. Optional:
   • Generate embeddings → insert doc.embedding

Requirements:

• Idempotent
• Upsert keyed by (repo, git_sha, path)
• Historical versions preserved

---

6. MCP Server (Next.js)

Location:

/app/api/mcp/route.ts

Transport:

• MCP Streamable HTTP

Next.js acts as a stateless façade over Postgres.

---

7. MCP Tools

7.1 doc.search

Input:

{
  "query": "kerberos optional kdc",
  "scope": "stable",
  "limit": 8
}

Baseline SQL:

SELECT d.doc_id, d.title, d.uri, s.git_sha
FROM doc.document d
JOIN doc.source s USING (source_id)
WHERE
  (d.lifecycle = 'stable' OR $scope = 'all')
  AND (d.title ILIKE $q OR d.content_text ILIKE $q)
ORDER BY d.updated_at DESC
LIMIT $limit;

Returns:

{
  "results": [
    { "doc_id": "...", "title": "...", "uri": "...", "git_sha": "..." }
  ]
}

---

7.2 doc.get

Input:

{ "doc_id": "..." }

SQL:

SELECT d.*, s.git_sha
FROM doc.document d
JOIN doc.source s USING (source_id)
WHERE d.doc_id = $1;

Returns full document with citation metadata.

---

7.3 doc.list_runbooks

Filtered by:

• Path prefix
• Tag field (future enhancement)

---

8. Vector Search (Phase 2)

Uses pgvector inside knoe-db.

Query example:

WITH ranked AS (
  SELECT c.doc_id,
         1 - (e.embedding <=> $query_embedding) AS score
  FROM doc.embedding e
  JOIN doc.chunk c USING (chunk_id)
  JOIN doc.document d USING (doc_id)
  WHERE d.lifecycle = 'stable'
  ORDER BY e.embedding <=> $query_embedding
  LIMIT 20
)
SELECT DISTINCT doc_id FROM ranked;

Important:

• Vector search returns doc_id only.
• Final filtering and citations always use authoritative document table.

Vector index is derived, not core.

---

9. Security Model

• Next.js handles authentication (OAuth/session)
• MCP endpoint validates user
• Database role is read-only
• No filesystem reads
• No direct git access from MCP
• NetworkPolicy: only Next.js → Postgres
• No shell execution

---

10. Versioning Model

Every answer includes:

• doc://doc/{doc_id}
• git_sha
• lifecycle

Stable answers reference only stable documents.

Reproducibility guarantee:

Given a git_sha, the answer corpus is reconstructible.

---

11. Efficiency Rationale

• Single authoritative core (Postgres)
• No language-dependent core logic
• Native integration with Next.js ecosystem
• Vector search does not introduce a new source of truth
• Policy enforced at SQL layer
• Backup/restore handled by CNPG + Barman

---

Final Principle

Postgres owns knowledge.
Next.js exposes it.
Everything else is replaceable.