prole/docs/knoe-db-documentation-mcp-architecture.md

# Knoe-DB Documentation MCP Architecture

**Design: Postgres-Core, Next.js Edge**

------------------------------------------------------------------------

## 1. Objective

Implement a Documentation MCP Server integrated into **Prol.app
(Next.js)** where:

-   **Postgres (knoe-db) is the only authoritative core**
-   Next.js provides the MCP interface and UI
-   Vector search is optional and derived
-   Ingestion is idempotent and Git-versioned
-   All responses are citation-grounded and reproducible

There is no Python core and no secondary business-logic layer.\
**The database owns truth, provenance, and policy.**

------------------------------------------------------------------------

## 2. High-Level Architecture

    Git Repo (docs branch)
            ↓
    CI → doc-manifest.json (git_sha, files, hashes)
            ↓
    Ingestion Worker (Node k8s Job)
            ↓
    Postgres (knoe-db)  ← authoritative core
            ↓
    Next.js (Prol.app)
       ├── UI (/docs, /search)
       └── MCP Server (/api/mcp)

Optional vector indexing:

    Postgres → embedding worker → pgvector (same DB)

------------------------------------------------------------------------

## 3. Core = Postgres Schema

Schema name: `doc`

Postgres is the authoritative knowledge store.

### 3.1 `doc.source`

Tracks document origin and versioning.

  Column         Type          Notes
  -------------- ------------- -----------------
  source_id      uuid pk       
  repo           text          git repository
  git_sha        text          commit hash
  path           text          file path
  ingested_at    timestamptz   
  content_hash   text          integrity check

------------------------------------------------------------------------

### 3.2 `doc.document`

Canonical document metadata.

  Column            Type          Notes
  ----------------- ------------- ---------------------------------
  doc_id            uuid pk       
  source_id         uuid fk       references doc.source
  title             text          
  uri               text unique   `doc://doc/{doc_id}`
  lifecycle         text          `stable`, `draft`, `deprecated`
  confidentiality   text          `internal`, `restricted`
  content_text      text          full markdown content
  updated_at        timestamptz   

------------------------------------------------------------------------

### 3.3 `doc.chunk`

Chunked content for search and embeddings.

  Column        Type      Notes
  ------------- --------- -------------
  chunk_id      uuid pk   
  doc_id        uuid fk   
  ordinal       int       chunk order
  text          text      
  token_count   int       

------------------------------------------------------------------------

### 3.4 `doc.embedding` (Optional)

Requires pgvector.

  Column       Type           Notes
  ------------ -------------- -------
  chunk_id     uuid pk        
  embedding    vector(1536)   
  model        text           
  indexed_at   timestamptz    

------------------------------------------------------------------------

### 3.5 `doc.link`

Semantic relationships between documents.

  Column     Type
  ---------- -------------------------------------------------
  from_doc   uuid
  to_doc     uuid
  relation   text (`applies_to`, `supersedes`, `references`)

------------------------------------------------------------------------

## 4. Policy Enforcement

Default rule:

> Only `stable` documents are searchable unless explicitly overridden.

Enforcement options:

-   SQL WHERE clauses in MCP queries (initial phase)
-   Row Level Security (future phase)

Confidentiality gating:

-   MCP layer passes `user_role`
-   Queries filter by `confidentiality <= role_level`

Every answer must include:

-   `doc_id`
-   `uri`
-   `git_sha`

The database guarantees provenance.

------------------------------------------------------------------------

## 5. Ingestion Pipeline

### 5.1 Trigger

Git push to docs branch triggers CI.

### 5.2 CI Output

`doc-manifest.json`

``` json
{
  "repo": "knoe-db",
  "git_sha": "abc123",
  "files": [
    { "path": "runbooks/kerberos.md", "hash": "..." }
  ]
}
```

### 5.3 Ingestion Worker (Node.js, Kubernetes Job)

Process:

1.  Read manifest
2.  For each file:
    -   Compute content hash
    -   Upsert `doc.source`
    -   Upsert `doc.document`
    -   Chunk content → insert `doc.chunk`
3.  Optional:
    -   Generate embeddings → insert `doc.embedding`

Requirements:

-   Idempotent
-   Upsert keyed by `(repo, git_sha, path)`
-   Historical versions preserved

------------------------------------------------------------------------

## 6. MCP Server (Next.js)

Location:

    /app/api/mcp/route.ts

Transport:

-   MCP Streamable HTTP

Next.js acts as a stateless façade over Postgres.

------------------------------------------------------------------------

## 7. MCP Tools

### 7.1 `doc.search`

Input:

``` json
{
  "query": "kerberos optional kdc",
  "scope": "stable",
  "limit": 8
}
```

Baseline SQL:

``` sql
SELECT d.doc_id, d.title, d.uri, s.git_sha
FROM doc.document d
JOIN doc.source s USING (source_id)
WHERE
  (d.lifecycle = 'stable' OR $scope = 'all')
  AND (d.title ILIKE $q OR d.content_text ILIKE $q)
ORDER BY d.updated_at DESC
LIMIT $limit;
```

Returns:

``` json
{
  "results": [
    { "doc_id": "...", "title": "...", "uri": "...", "git_sha": "..." }
  ]
}
```

------------------------------------------------------------------------

### 7.2 `doc.get`

Input:

``` json
{ "doc_id": "..." }
```

SQL:

``` sql
SELECT d.*, s.git_sha
FROM doc.document d
JOIN doc.source s USING (source_id)
WHERE d.doc_id = $1;
```

Returns full document with citation metadata.

------------------------------------------------------------------------

### 7.3 `doc.list_runbooks`

Filtered by:

-   Path prefix
-   Tag field (future enhancement)

------------------------------------------------------------------------

## 8. Vector Search (Phase 2)

Uses pgvector inside knoe-db.

Query example:

``` sql
WITH ranked AS (
  SELECT c.doc_id,
         1 - (e.embedding <=> $query_embedding) AS score
  FROM doc.embedding e
  JOIN doc.chunk c USING (chunk_id)
  JOIN doc.document d USING (doc_id)
  WHERE d.lifecycle = 'stable'
  ORDER BY e.embedding <=> $query_embedding
  LIMIT 20
)
SELECT DISTINCT doc_id FROM ranked;
```

Important:

-   Vector search returns `doc_id` only.
-   Final filtering and citations always use authoritative document
    table.

Vector index is derived, not core.

------------------------------------------------------------------------

## 9. Security Model

-   Next.js handles authentication (OAuth/session)
-   MCP endpoint validates user
-   Database role is read-only
-   No filesystem reads
-   No direct git access from MCP
-   NetworkPolicy: only Next.js → Postgres
-   No shell execution

------------------------------------------------------------------------

## 10. Versioning Model

Every answer includes:

-   `doc://doc/{doc_id}`
-   `git_sha`
-   `lifecycle`

Stable answers reference only stable documents.

Reproducibility guarantee:

Given a `git_sha`, the answer corpus is reconstructible.

------------------------------------------------------------------------

## 11. Efficiency Rationale

-   Single authoritative core (Postgres)
-   No language-dependent core logic
-   Native integration with Next.js ecosystem
-   Vector search does not introduce a new source of truth
-   Policy enforced at SQL layer
-   Backup/restore handled by CNPG + Barman

------------------------------------------------------------------------

## Final Principle

**Postgres owns knowledge.\
Next.js exposes it.\
Everything else is replaceable.**