# KnowHub — Product Design Document

**Version:** 4.0 | **Date:** 2026-03-20 | **Status:** Single Source of Truth
**Supersedes:** All prior design documents (V1-V3, operations-design.md, v1-operations-final.md)

---

## 1. Product Vision

KnowHub is a knowledge distillation engine for AI agents. It transforms raw agent experience into dense, reusable expertise — through a lifecycle of extraction, structuring, and crystallization, with human oversight at every transition.

**One-liner:** From conversations to expertise. Not storage — distillation.

**Three audiences:**
- **Agents** — read/write memories, consume knowledge, execute skills via API.
- **Owners** — oversee, curate, and correct their agent's knowledge through the web console.
- **Teams** — share knowledge and skills across agents through shared vaults.

**Core value proposition:** Agents that learn from experience and get better over time. KnowHub turns raw agent activity into refined institutional expertise, with human judgment at every transition.

---

## 2. The Distillation Lifecycle

```
CONVERSATION  (raw, ephemeral, high-volume, low-density)
      |
      | EXTRACT  (agent leads, human reviews)
      v
MEMORY  (observations, preferences, facts — medium density)
      |
      | STRUCTURE  (human leads, AI assists)  [M2]
      v
KNOWLEDGE  (reference material — high density, reusable)
      |
      | CRYSTALLIZE  (either initiates, human approves)  [M2]
      v
SKILL  (executable procedures — highest density, actionable)
```

Each transition compresses information. Knowledge density increases at every level. Currently, the EXTRACT transition (conversation to memory) is agent-handled. The STRUCTURE and CRYSTALLIZE transitions are planned for M2.

---

## 3. Core Concepts

### 3.1 Content Types

| Concept | Definition | Storage | Color |
|---------|-----------|---------|-------|
| **Vault** | Container for one agent's knowledge. Private (lock icon) or shared (globe icon). | Git repository | — |
| **Memory** | Short-form agent observation or learning. Agent-authored, human-stewarded. | Issue (no `type:conversation` label) | Amber `#d29922` |
| **Conversation** | Agent-human chat log. A session of messages. Read-only content. | Issue (`type:conversation` label) + comments | — |
| **Knowledge** | Reference material with optional YAML frontmatter metadata. Files rendered as Markdown (Preview/Raw toggle). | File in `/knowledge/` | Blue `#58a6ff` |
| **Skill** | Agent capability following the Agent Skills standard. A directory (SKILL.md entry + templates/ + scripts/ + references/ + assets/) or a single .md file. Rendered as Markdown. | Directory or file in `/skills/` | Green `#3fb950` |
| **Comment** | Owner annotation on a memory. | Issue comment | — |

### 3.2 Vault Types

All vaults are repositories. The distinction is `isPrivate`:
- **Private vault** (lock icon) — one agent, one owner. Contains memories, conversations, knowledge, skills.
- **Shared vault** (globe icon) — public repository. Primarily knowledge and skills shared across agents.

Both appear in the same Vault type and share the same detail page. The sidebar shows them in separate sections: "MY VAULTS" and "SHARED VAULTS".

### 3.3 Skill Model

A Skill is either:
1. **Directory-based** — a folder in `/skills/` containing a `SKILL.md` entry point plus optional subdirectories (`templates/`, `scripts/`, `references/`, `assets/`). Displayed as a tree structure in the detail view.
2. **Single-file** — a standalone `.md` file in `/skills/`.

Skills are rendered as Markdown using the Preview/Raw toggle (MarkdownViewer component). YAML frontmatter is parsed for `name`, `description`, and `category`. All content below frontmatter is the skill's instructions.

**SKILL.md format:**
```yaml
---
name: skill-name
description: |
  What this skill does and when to use it.
allowed-tools: []
metadata: {}
---
# Skill Instructions

Step-by-step markdown instructions the agent follows.
```

### 3.4 Knowledge Model

Knowledge documents are files in `/knowledge/`. They support nested folders. Files are rendered with the MarkdownViewer (Preview/Raw toggle). When YAML frontmatter is present (`title`, `description`, `tags`, `type`), the DocumentCard shows metadata-first cards with description, type badge, and tag pills. Files without frontmatter show filename and size.

### 3.5 Forbidden Terms

These Git terms must NEVER appear in the UI:

| Forbidden | Use Instead |
|-----------|-------------|
| Repository / Repo | Vault |
| Commit | Save |
| Issue | Memory / Conversation |
| Label | Tag |
| Clone / Fork / Pull / Push | _(not exposed)_ |
| Git | _(never mention)_ |

---

## 4. Information Architecture

### 4.1 Sitemap

```
/login
/ (Home — vault grid + activity stream + stats)
/vaults/[slug] (vault detail — tabs below)
  Overview | Memories (N) | Conversations (N) | Knowledge | Skills
/vaults (vault list, used as overflow from sidebar)
/search
/settings (tabs: Account, Security, Vaults, Notifications, AI Configuration)
```

**Removed routes** (exist as files but are not linked in navigation):
- `/activity` — merged into Home page
- `/spaces`, `/spaces/new`, `/spaces/[slug]` — replaced by unified vault concept

### 4.2 Navigation

**Left sidebar (collapsible):**
- KnowHub logo
- Search bar (Cmd+K)
- Home
- MY VAULTS section — private vaults listed with lock icon (up to 5, with "View all" overflow)
- SHARED VAULTS section — public vaults listed with globe icon (up to 5, with "View all" overflow)
- User info (avatar, login, server)
- Settings
- Sign out

**Vault detail:** Tab bar — Overview | Memories (N) | Conversations (N) | Knowledge | Skills

---

## 5. Page Specifications

### 5.1 Home Page (/)

**Purpose:** Dashboard answering "what is my agent up to?" at a glance.

**Content:**
- Greeting (time-based)
- Attention alerts (fading memories, derived from vault data)
- Two-column layout:
  - Left: "My Vaults" grid (private) + "Shared Vaults" grid (public)
  - Right: Stats cards (total memories, documents, skills, avg health) + Recent Activity stream
- Activity events shown inline (no separate Activity page)

### 5.2 Vault Detail — Overview Tab

**Purpose:** Vault-level summary.

**Content:**
- VaultOverviewPanel component
- Attention alerts for this vault (fading memories)
- Recent activity events filtered to this vault
- Stats: memory count, document count, skill count

### 5.3 Vault Detail — Memories Tab

**Layout:** Filter sidebar (left) + memory stream (center) + detail panel (right, on selection).

**Memory card shows:** Title, content preview, importance (0-1), status badge (active/fading/archived), tags, source, timestamp.

**Filter sidebar:** Dynamic from data — filter by status, tags, sources, sort (newest/oldest/importance).

**Memory detail panel shows:**
- Full content
- Tags (editable via TagEditor — add/remove tags)
- Importance score, status, source
- Created date
- Comments thread (owner annotations)

**Write operations:**
- **Comment on memory** — text input at bottom of detail panel
- **Delete memory** — trash icon with confirmation modal
- **Add/remove tags** — TagEditor component (creates labels on the git service)

**Memory Hygiene (AI):** When AI is configured and the feature is enabled, a "Check Hygiene" button triggers duplicate and contradiction detection across all memories. Results shown in a collapsible panel.

### 5.4 Vault Detail — Conversations Tab

**Layout:** Session list (left) + ConversationViewer (right, on selection).

**Session list:** Issues with `type:conversation` label. Shows title, date, message count.

**ConversationViewer:** Loads issue comments, parses `role: user` / `role: assistant` header to determine message role. Alternating alignment for agent vs human messages.

**Write operations:**
- **Delete conversation** — confirmation modal
- **Add/remove tags** — TagEditor component
- **Summarize (AI)** — generates a summary + auto-tags via LLM, shown inline

### 5.5 Vault Detail — Knowledge Tab

**Layout:** Search bar (top) + folder browser + document cards (main) + DocumentViewer (on selection).

**Document card shows:** Filename, size, type badge, last modified. When YAML frontmatter exists: title, description, type badge, tag pills (metadata-first rendering via `parseFrontmatter`).

**DocumentViewer shows:**
- File metadata header (name, path, size, last modified)
- Markdown content with Preview/Raw toggle (MarkdownViewer)
- Edit mode (textarea with Save/Cancel)
- Delete button

**Write operations:**
- **Upload file** — FileUploadZone component (file picker). Optional AI metadata extraction on upload (Smart Upload).
- **Edit file** — inline editing, save commits to git service
- **Delete file** — confirmation modal

**Staleness Check (AI):** When AI is configured, knowledge documents can be checked for outdated content (temporal references, version numbers). Stale indicators shown on cards.

### 5.6 Vault Detail — Skills Tab

**Layout:** Skill card grid (main) + SkillDetail panel (on selection).

**Skill card shows:** Name, description, category badge, last modified.

**SkillDetail shows:**
- Header: name, description, category
- File tree (for directory-based skills — lists files in templates/, scripts/, etc.)
- SKILL.md content rendered via MarkdownViewer (Preview/Raw toggle)
- Edit mode (edit SKILL.md content, save commits to git service)
- Delete button
- Analyze button (AI)

**Write operations:**
- **Upload file** — FileUploadZone component
- **Edit skill** — edit SKILL.md content, save = commit
- **Delete skill** — confirmation modal

**Skill Analyzer (AI):** Scores skill quality on four dimensions (clarity, completeness, specificity, efficiency, each 1-5). For any score below 4, provides a concrete fix suggestion. Results cached in sessionStorage.

---

## 6. Write Operations Summary

| # | Operation | Scope | Trigger | API |
|---|-----------|-------|---------|-----|
| 1 | **Delete** | All content types | Trash icon + confirmation modal | `PATCH issues/{n}` (close + label) or `DELETE contents/{path}` |
| 2 | **Upload file** | Knowledge, Skills | FileUploadZone + optional AI metadata extraction | `PUT contents/{path}` with base64 content |
| 3 | **Edit file** | Knowledge docs, Skills (SKILL.md) | Edit button → textarea → Save | `PUT contents/{path}` with `sha` |
| 4 | **Comment on memory** | Memories | Text input in detail panel | `POST issues/{n}/comments` |
| 5 | **Add/remove tags** | Memories, Conversations | TagEditor component | `POST repos/{o}/{r}/labels` + `POST issues/{n}/labels` / `DELETE issues/{n}/labels/{name}` |
| 6 | **Summarize conversation** | Conversations | "Summarize" button (AI) | `POST /api/ai/invoke` (action: summarize-conversation) |
| 7 | **Share vault** | Vault level | Share button → ShareVaultModal | Displays connection config (server URL + token) for copy |

**Explicitly not in the UI:**
- Create memory (agents author memories)
- Edit memory content (erases provenance; use comments)
- New Document / New Skill from scratch (upload only)
- Bulk operations
- Move/rename file

---

## 7. Authentication

**Login page (`/login`):**
- Server URL input (default: `https://git.clawmem.ai/api/v3`)
- Two auth methods:
  - **Token** — paste a personal access token
  - **Device Flow** — OAuth device code flow (start flow → copy user code → authorize in browser → poll for token)

**Architecture:**
- Auth state stored in `localStorage` as `knowhub_auth: { isAuthenticated, serverUrl, token, user? }`
- All API calls go through Next.js proxy at `/api/proxy/*`
- Proxy receives auth via custom headers: `X-KnowHub-Server` (server URL) + `Authorization` (token)
- Proxy forwards to the target git service
- Works with any GitHub-compatible API: GitHub.com, Gitea, Forgejo, agent-git-service

**Auth API routes:**
- `POST /api/auth/validate` — validate token against git service
- `POST /api/auth/device-code` — start device flow
- `POST /api/auth/device-token` — poll for device flow token

---

## 8. AI Features

All AI features are optional. KnowHub works fully without LLM configuration.

### 8.1 Configuration

Located at **Settings > AI Configuration** (AIConfigPanel component):

- Provider selector: OpenAI / Anthropic / Custom
- API Key (password input)
- Model name
- Base URL
- Feature toggles (each can be individually enabled/disabled):
  - Smart Upload
  - Skill Analyzer
  - Memory Hygiene
  - Content Classification (Skill Identifier)
  - Staleness Check

**Defaults:** Provider = Custom, Model = `qwen3.5-plus`, Base URL = `https://dashscope-intl.aliyuncs.com/compatible-mode/v1` (Qwen 3.5 Plus via DashScope Singapore).

**Provider presets:**
- OpenAI: `gpt-4o-mini`, `https://api.openai.com/v1`
- Anthropic: `claude-haiku-4-5-20251001`, `https://api.anthropic.com`
- Custom: `qwen3.5-plus`, DashScope URL

Config stored in `localStorage` (key: `knowhub_ai_config`).

### 8.2 Implemented AI Features

| # | Feature | Trigger | What It Does |
|---|---------|---------|-------------|
| 1 | **Smart Upload** | Automatic on file upload | Extracts title, description, tags, classification (skill vs knowledge), confidence, reasoning. Pre-fills Import Review. |
| 2 | **Skill Analyzer** | Manual — "Analyze" button on SkillDetail | Scores clarity/completeness/specificity/efficiency (1-5). Suggests fixes for scores below 4. Cached in sessionStorage. |
| 3 | **Memory Hygiene** | Manual — "Check Hygiene" button on Memories tab | Detects duplicate and contradictory memories. Shows grouped results with explanations. |
| 4 | **Skill Identifier** | Automatic — part of Smart Upload | Classifies uploaded content as skill or knowledge with confidence level and reasoning. |
| 5 | **Staleness Check** | Manual — triggered per document on Knowledge tab | Flags outdated content based on date references, version numbers, tool references. |
| 6 | **Conversation Summarize** | Manual — "Summarize" button per conversation | Generates a 2-3 sentence summary + 3-5 suggested tags. Tags can be applied to the conversation. |

### 8.3 Architecture

```
Browser (no AI keys in client code)
  → POST /api/ai/invoke  { action, payload }
    → headers: X-KnowHub-AI-Provider, X-KnowHub-AI-Key, X-KnowHub-AI-Model, X-KnowHub-AI-BaseURL
      → LLM Provider API (OpenAI-compatible or Anthropic)
        → Response parsed as JSON
          → { ok: true, data: {...}, tokens_used: N }
```

- Single unified AI route: `POST /api/ai/invoke`
- Action-based dispatch with prompt templates (one template per action)
- Supports OpenAI-compatible API (used for OpenAI, Qwen, and any custom provider) and Anthropic API
- 15-second timeout per request
- JSON response extraction with fallback (direct parse → code fence extraction → brace matching)
- On LLM failure, features degrade silently. Background features simply do not show results.

**Fallback behavior (no LLM configured):**
- Smart Upload: metadata fields blank, filename as title
- Skill Analyzer: "Analyze" button hidden
- Memory Hygiene: "Check Hygiene" button hidden
- Staleness Check: no content analysis
- Conversation Summarize: "Summarize" button hidden

---

## 9. Technical Architecture

```
Browser → Next.js App (Next.js 16, React 19, TypeScript)
  ├── Pages + Components (App Router)
  ├── /api/proxy/[...path] → Git Service (GitHub-compatible API)
  ├── /api/auth/*           → Auth validation, device flow
  └── /api/ai/invoke        → LLM Provider (optional)

Git Service (agent-git-service, Gitea, GitHub)
  ├── Issues → Memories (no type:conversation label)
  │             Conversations (type:conversation label)
  ├── Issue Comments → Memory comments, Chat messages
  ├── Files in /knowledge/ → Knowledge documents
  ├── Files in /skills/ → Skills (directory or single file)
  └── Labels → Tags + type metadata
```

### 9.1 Tech Stack

- **Framework:** Next.js 16.2.0 + React 19.2.4 + TypeScript 5
- **Styling:** Tailwind CSS v4 (dark theme)
- **Icons:** lucide-react
- **Markdown:** react-markdown 10
- **Testing:** Puppeteer (E2E tests against real git service)
- **Utility:** clsx (class merging)

### 9.2 Data Mapping

| UI Concept | Git Concept | API Path |
|-----------|-------------|----------|
| Memory | Issue (no `type:conversation` label) | `/repos/{o}/{r}/issues` filtered |
| Conversation | Issue (`type:conversation` label) | `/repos/{o}/{r}/issues` filtered |
| Chat message | Issue comment (parsed by role header) | `/repos/{o}/{r}/issues/{n}/comments` |
| Memory comment | Issue comment | `/repos/{o}/{r}/issues/{n}/comments` |
| Knowledge item | File in `/knowledge/` (recursive) | `/repos/{o}/{r}/contents/knowledge/{path}` |
| Skill | Directory or file in `/skills/` | `/repos/{o}/{r}/contents/skills/{path}` |
| Tag | Label | `/repos/{o}/{r}/labels` |
| Vault | Repository | `/repos/{o}/{r}` |

### 9.3 Key TypeScript Types

See `frontend/src/lib/types.ts` for full definitions. Key types:

- **Memory** — id, title, content, status (active/fading/archived), importance (0-1), tags[], source, agent, commentCount
- **Document** — name, path, folder, content, size, type (markdown/pdf/image/text/other), lastModified
- **Skill** — name, path, entryPath (to SKILL.md), description, category, content, files[] (tree), lastModified
- **Vault** — id, owner, name, fullName, description, isPrivate, memoryCount, documentCount, skillCount, health, fadingCount
- **AIConfig** — provider, apiKey, model, baseUrl, features (5 boolean toggles)

### 9.4 Vault Slug Encoding

Vaults are identified by `owner--repo` slugs in URLs: `encodeVaultSlug("alice", "vault-1")` yields `alice--vault-1`. Decoded with `decodeVaultSlug`.

---

## 10. Visual Design Language

- **Dark theme** — primary background, all UI surfaces
- **Typography:** Geist Sans (UI) / Geist Mono (code, metadata)
- **Border radius:** 8px (cards), 6px (inputs), 4px (badges)

**Content type colors:**

| Type | Color | Hex |
|------|-------|-----|
| Memory | Amber | `#d29922` |
| Knowledge | Blue | `#58a6ff` |
| Skill | Green | `#3fb950` |

**Key components:** ConfirmationModal, Toast, FileUploadZone, MarkdownViewer (Preview/Raw toggle), TagEditor, ShareVaultModal, ConversationViewer, MemoryFilterSidebar, DocumentCard (metadata-first when frontmatter present), SkillCard, SkillDetail (tree + markdown + edit + analyze), VaultCard, AIConfigPanel.

---

## 11. Implementation Status

### M1: Foundation — SHIPPED

| Feature | Status |
|---------|--------|
| Login + Auth (token + device flow, default git.clawmem.ai) | Done |
| Home page (vault grid + activity stream + stats) | Done |
| Sidebar: Home, MY VAULTS, SHARED VAULTS, Settings, Sign out | Done |
| Vault detail (5 tabs: Overview, Memories, Conversations, Knowledge, Skills) | Done |
| Memories tab (filter sidebar + detail panel + comments) | Done |
| Conversations tab (session list + ConversationViewer) | Done |
| Knowledge tab (folder browser + DocumentViewer + Markdown Preview/Raw) | Done |
| Skills tab (card grid + SkillDetail with tree + Markdown Preview/Raw) | Done |
| Delete (all content types, confirmation modal) | Done |
| Upload file (Knowledge + Skills, FileUploadZone) | Done |
| Edit file (Knowledge docs + SKILL.md, save = commit) | Done |
| Comment on memory | Done |
| Tag management (add/remove tags on memories and conversations) | Done |
| Share vault (ShareVaultModal, token sharing) | Done |
| Conversation summarize (AI summary + auto-tags) | Done |
| AI Configuration (Settings tab: provider/key/model/baseUrl + feature toggles) | Done |
| Smart Upload (AI metadata extraction on file upload) | Done |
| Skill Analyzer (quality scoring: clarity/completeness/specificity/efficiency) | Done |
| Memory Hygiene (duplicate/contradiction detection) | Done |
| Staleness Check (flag outdated knowledge docs) | Done |
| Skill Identifier (classify content as skill vs knowledge) | Done |
| API proxy with custom auth headers | Done |
| Unified AI route (POST /api/ai/invoke) | Done |
| E2E test suite (Puppeteer, tested against git.clawmem.ai) | Done |
| Zero mock data — all real data from git service | Done |

### M2: Knowledge Distillation Engine — NOT STARTED

| Feature | Description |
|---------|-------------|
| **Consolidate Memories** | Select related memories → AI generates structured knowledge doc → split-view editor → save to /knowledge/ |
| **Extract from Conversation** | AI scans conversation → surfaces potential memories agent missed → one-click save |
| **Promote to Skill** | Procedural knowledge doc → AI converts to SKILL.md format → review and save to /skills/ |
| Distillation Dashboard | Overview tab evolves: pipeline visualization, distillation health score, suggestion engine |

---

_This document is the single source of truth for KnowHub product design. It reflects the actual shipped state of the codebase as of 2026-03-20._