# KnowHub — Knowledge Distillation Design **Version:** 1.0 | **Date:** 2026-03-19 | **Status:** Design Proposal **Extends:** DESIGN.md (V3), ai-features-design.md (V1.1) --- ## 1. Revised Product Vision **Old:** "KnowHub is a memory and knowledge management console for AI agents." **New:** KnowHub is a knowledge distillation engine for AI agents. It transforms raw agent experience into dense, reusable expertise — through a lifecycle of extraction, structuring, and crystallization, with human oversight at every transition. **One-liner:** From conversations to expertise. Not storage — distillation. **The analogy:** Memory is knowing how to write papers that get published and cited — distilled from countless papers you have written. Long context is cramming the night before an exam. KnowHub produces the former, not the latter. **Core value proposition (revised):** Agents that learn from experience and get better over time — not agents that accumulate files. KnowHub turns the chaff of daily agent activity into the wheat of institutional expertise, with human judgment at every transition. --- ## 2. The Distillation Lifecycle ### 2.1 The Four Levels ``` CONVERSATION (raw, ephemeral, high-volume, low-density) ~2000 tokens per session, mostly noise | | EXTRACT v MEMORY (observations, preferences, facts — medium density) ~50 tokens each, atomic, agent-authored | | STRUCTURE v KNOWLEDGE (reference material — high density, reusable) ~300 tokens each, organized by topic, human-curated | | CRYSTALLIZE v SKILL (executable procedures — highest density, actionable) ~300 tokens each, agent-executable, human-approved ``` Each transition is a compression. Knowledge density (actionable information per token) increases at every level. This maps directly to the Knowledge Activation research: the AKU concept is the end state of this pipeline. ### 2.2 Who Does What at Each Transition | Transition | Agent Role | Human Role | AI Console Role | |------------|-----------|------------|-----------------| | **EXTRACT** (Conversation -> Memory) | Primary author. Agent writes memories during sessions. | Reviewer. Scans, corrects, deletes bad extractions. | Quality auditor. "3 potential memories found in this conversation that the agent missed." | | **STRUCTURE** (Memory -> Knowledge) | Not involved. | Initiator. Selects memories, triggers consolidation, reviews output. | Draft author. Produces structured knowledge doc from selected memories. Human edits and saves. | | **CRYSTALLIZE** (Knowledge -> Skill) | Pattern detector. Can suggest "this looks like a repeated procedure." | Decision maker. Approves promotion, edits skill parameters. | Restructurer. Converts procedural knowledge into SKILL.md format with frontmatter, steps, parameters. | **Key insight from the discussion:** The agent and human have different roles at each transition. The product must make these roles visible and the handoffs clean. - EXTRACT: Agent leads, human reviews. The agent is the writer; the human is the editor. - STRUCTURE: Human leads, AI assists. The human is the curator; the AI is the librarian. - CRYSTALLIZE: Either can initiate, but human approves. The AI is the formatter; the human is the quality gate. ### 2.3 The Density Multiplier | Level | Avg Tokens | Knowledge Density | Ratio | |-------|-----------|-------------------|-------| | Conversation | ~2000 | 1x (baseline) | — | | Memory | ~50 | 8-10x | 40:1 compression | | Knowledge | ~300 | 15-20x | 7:1 vs conversation | | Skill | ~300 | 25-30x | 7:1 vs conversation, but actionable | A vault with 100 conversations (200,000 tokens of raw material) should distill down to ~500 memories (25,000 tokens), ~30 knowledge docs (9,000 tokens), and ~10 skills (3,000 tokens). That 3,000 tokens of skills encodes more capability than the original 200,000 tokens of conversation. That is distillation. --- ## 3. Three Distillation Features ### 3.1 Feature: Consolidate Memories **What it does:** Transforms a set of related memories into a structured knowledge document. This is the STRUCTURE transition — the biggest gap in the current design. **Example:** An agent has accumulated 47 memories about a user's coding preferences: "User prefers functional style," "User dislikes classes in Python," "User wants type hints on all public functions," "User prefers pytest over unittest," etc. The user selects these (or filters by tag "coding-preferences"), clicks Consolidate, and the AI produces a structured "Coding Standards" knowledge document with sections, organized by topic. **Who triggers it:** Human. Always intentional, never automatic. **UI placement:** Two entry points: 1. **Memories tab** — multi-select memories (checkboxes appear on hover), then a "Consolidate" button appears in the action bar. Also available after applying a tag filter: "Consolidate filtered memories." 2. **Distillation Dashboard** — suggestion cards: "23 memories tagged 'preferences' could become a Knowledge document." Click to pre-select and enter the consolidation flow. **The flow:** ``` 1. User selects memories (multi-select or filter) 2. Clicks "Consolidate into Knowledge" 3. AI generates a structured document: - Extracts themes from the memories - Groups related facts - Removes redundancy - Adds frontmatter (title, description, tags, type) - Writes in agent-consumable style (no rhetorical scaffolding) 4. User sees a split view: Left: the source memories (scrollable, with highlights showing which parts were used) Right: the generated knowledge document (editable) 5. User edits, adjusts, then clicks "Save to Knowledge" 6. Document saved to /knowledge/ with frontmatter 7. Source memories get a "consolidated" tag linking to the new doc (memories are NOT deleted — they retain provenance) ``` **AI prompt (brief):** "Given these memories, create a structured knowledge document optimized for agent consumption. Group related facts under clear headings. Eliminate redundancy. Use concrete, imperative language. Include YAML frontmatter with name, description, tags, and type. The document should be a single authoritative reference that replaces the need to scan individual memories on this topic." **API route:** `POST /api/ai/consolidate-memories` - Input: `{ memories: {id, title, content}[], suggestedTitle?: string }` - Output: `{ document: string, frontmatter: {name, description, tags[], type}, sourceMapping: {memoryId, usedIn: string}[] }` **Fallback (no LLM):** Button is hidden. Feature does not appear. Memories tab works as before. ### 3.2 Feature: Extract from Conversation **What it does:** Surfaces potential memories from a conversation that the agent did not extract on its own. This is the EXTRACT transition — improving its coverage. **Example:** User reviews a conversation where they discussed deployment procedures with the agent. The agent only extracted one memory ("User deploys to staging first"). But the conversation also contains "We use GitHub Actions for CI," "Production deploys require two approvals," and "Rollback procedure is to revert the last merge." The AI surfaces these three as suggestions. **Who triggers it:** Semi-automatic. When a user opens a conversation, the AI scans it in the background. A subtle banner appears: "3 potential memories found." Manual trigger also available: "Scan for memories" button. **UI placement:** Conversation detail view. - Banner at top: "{N} potential memories found in this conversation" with "Review" link - Clicking "Review" opens a right drawer showing each suggested memory as a card - Each card has: suggested title, suggested content, "Save as Memory" button, "Dismiss" button - Saved memories get a `source:conversation:{id}` tag for provenance **The flow:** ``` 1. User opens a conversation in the Conversations tab 2. Background: AI scans the conversation for extractable knowledge 3. Banner appears (or not, if nothing found) 4. User clicks "Review" 5. Right drawer shows suggested memories as editable cards 6. User saves the ones they want (one-click each) 7. Saved memories appear in the Memories tab with source attribution ``` **AI prompt (brief):** "Given this conversation between a human and an agent, identify facts, preferences, decisions, procedures, or constraints that would be valuable as standalone memories. For each, provide a concise title and content (under 80 tokens). Exclude things that are session-specific, already obvious, or likely already captured. Respond as JSON." **API route:** `POST /api/ai/extract-from-conversation` - Input: `{ messages: {role, content}[], existingMemories?: {title, content}[] }` - Output: `{ suggestions: {title, content, reasoning, confidence}[] }` **Fallback (no LLM):** Banner never appears. Conversations tab works as before. ### 3.3 Feature: Promote to Skill **What it does:** Converts a procedural knowledge document into a properly formatted SKILL.md with frontmatter, steps, parameters, and tool bindings. This is the CRYSTALLIZE transition. **Example:** A knowledge document titled "How We Generate Weekly Reports" describes a multi-step procedure: pull data from the database, aggregate by team, format as markdown table, post to Slack. The user clicks "Promote to Skill," and the AI restructures it into Agent Skills standard format with `name`, `description`, `parameters` (team, date_range, channel), numbered steps, and `allowed-tools`. **Who triggers it:** Human. The Distillation Dashboard may suggest candidates ("3 knowledge docs describe procedures that could become skills"), but the user decides. **UI placement:** Two entry points: 1. **Knowledge detail view** — "Promote to Skill" button in the action bar, next to Edit/Delete. Only visible when the document type is "Procedure" or when the AI classifies it as procedural. 2. **Distillation Dashboard** — suggestion cards: "This knowledge doc describes a procedure. Promote to Skill?" **The flow:** ``` 1. User views a procedural knowledge document 2. Clicks "Promote to Skill" 3. AI analyzes the document and generates a SKILL.md: - Extracts name, description from the content - Identifies parameters (inputs the procedure needs) - Structures steps as numbered instructions - Identifies tool bindings (APIs, commands mentioned) - Adds governance constraints if mentioned 4. User sees a split view: Left: original knowledge document Right: generated SKILL.md (editable, with frontmatter visible) 5. User reviews, edits parameters, adjusts steps 6. Clicks "Save as Skill" 7. Skill saved to /skills/ directory 8. Original knowledge doc gets a "promoted:skill:{name}" tag (knowledge doc is NOT deleted — it remains as reference) ``` **AI prompt (brief):** "Convert this knowledge document into an Agent Skills standard SKILL.md file. Extract: name, description, parameters (with types and whether required), step-by-step instructions (imperative, concrete, numbered), allowed-tools, and any constraints or conditions. Use the YAML frontmatter format. Each step should be one clear action. Remove explanatory text that is not actionable." **API route:** `POST /api/ai/promote-to-skill` - Input: `{ content: string, filename: string }` - Output: `{ skillMd: string, frontmatter: {name, description, parameters[], allowedTools[]}, stepCount: number }` **Fallback (no LLM):** "Promote to Skill" button hidden. Feature does not appear. --- ## 4. V1.1 Priority: Consolidate Memories ### 4.1 Why This Ships First **Consolidate Memories is the one feature that ships in V1.1.** The reasoning: 1. **It fills the biggest gap.** The current design has no path from Memory to Knowledge. Agents create memories. Humans can browse them. But there is no mechanism to synthesize 50 memories into one coherent reference document. This is the missing middle of the distillation pipeline. 2. **It creates immediate value.** Any vault with 20+ memories benefits. The user does not need to adopt new workflows — they already have memories. This feature activates existing content. 3. **It demonstrates the core thesis.** When a user sees 47 scattered memories about coding preferences compress into one clean 300-token reference document, they viscerally understand what distillation means. It is the "aha" moment for the product. 4. **It is self-contained.** Does not require changes to the agent API. Does not require conversation storage changes. Only needs: multi-select in the Memories tab, one new AI endpoint, a split-view editor, and a save-to-Knowledge flow. 5. **It feeds the next features.** Once users have more knowledge documents (created via consolidation), the Promote to Skill feature has more raw material. Consolidation primes the pump for the full lifecycle. ### 4.2 What Defers - **Extract from Conversation** defers to V1.2. Requires conversation content analysis, which is more complex (longer inputs, more nuanced extraction). Also, the agent already does extraction — this feature improves coverage, but the gap is smaller. - **Promote to Skill** defers to V1.2. Depends on having procedural knowledge documents, which consolidation helps create. Ship consolidation first to build the knowledge base that promotion draws from. - **Distillation Dashboard** defers to V1.2. Needs all three features active to be meaningful. Shipping the dashboard before the features exist shows problems without solutions. --- ## 5. How This Changes Existing Design ### 5.1 What Moves | Current | Change | Rationale | |---------|--------|-----------| | Overview tab shows stats (count of memories, knowledge, skills) | Add "distillation health" metrics: uncollected conversations, unconsolidated memory clusters, procedural knowledge that could become skills | The Overview becomes a distillation status view, not just a counter | | Memories tab is view/comment/delete only | Add multi-select mode and "Consolidate" action | Memories become the input to distillation, not just an archive | | Knowledge tab is upload/edit/delete only | Add "Promoted from memories" provenance badge; "Promote to Skill" action on procedural docs | Knowledge becomes a waypoint in the lifecycle, not a destination | | AI features (V1.1) are independent utilities | Consolidate Memories joins the feature set; all features reframed as distillation assistants | Features serve a unified purpose instead of being standalone tools | ### 5.2 What Gets Renamed | Old | New | Why | |-----|-----|-----| | "Overview" tab (on vault detail) | "Overview" (no change, but content changes) | Name is fine; content shifts from stats to distillation health | | AI features section title: "V1.1 AI Features" | "Distillation & AI Features" | Consolidation is the headliner; analysis features support it | The existing five AI features from ai-features-design.md remain unchanged. They are complementary: - **Smart Upload** helps at the Knowledge level (better metadata on entry) - **Skill Analyzer** helps at the Skill level (quality check after crystallization) - **Memory Hygiene** helps at the Memory level (cleaning before consolidation) - **Content Classification** helps at the Knowledge/Skill boundary (routing) - **Staleness Check** helps at the Knowledge level (maintenance) Consolidate Memories is the new sixth feature, and it is the only one that performs a *transition* between levels rather than improving quality within a level. ### 5.3 What Gets Added 1. **Multi-select mode on Memories tab.** Checkboxes on hover, select-all within filter, action bar with "Consolidate" button. 2. **Consolidation split-view.** Full-screen modal: source memories on left, generated document on right. Edit-in-place on the right. "Save to Knowledge" button. 3. **Provenance badges.** On Knowledge cards: "Consolidated from 23 memories" with link to source memories. On Memories: "Consolidated into [doc name]" tag. On Skills: "Promoted from [doc name]" tag. 4. **Distillation hints on Overview tab.** Three new attention alerts: - "47 memories tagged 'preferences' — consolidate into a reference doc?" - "Knowledge doc 'Deploy Process' describes a procedure — promote to skill?" - "12 conversations with no memories extracted" (V1.2, when Extract ships) 5. **New API routes.** `POST /api/ai/consolidate-memories` added to the AI Configuration module. Same architecture: browser -> Next.js API route -> LLM provider. ### 5.4 What Does NOT Change - Agent API. Agents still write memories and read knowledge/skills the same way. Distillation is a console-side concern. - Vault structure. Still Git repositories. Memories are still issues. Knowledge is still files in `/knowledge/`. Skills are still files in `/skills/`. - Navigation. Same sidebar, same tabs. No new top-level pages in V1.1. - Forbidden terms. Still no Git terminology in the UI. - Fallback behavior. Everything works without LLM configuration. Consolidation button is hidden if no LLM is configured. --- ## 6. Distillation Dashboard Spec (V1.2) **Ships in V1.2**, after the three distillation features are live. This section is the advance spec so the V1.1 Overview tab changes can be designed as a stepping stone. ### 6.1 Purpose Answer the question: "How healthy is my vault's knowledge pipeline?" Surface actionable suggestions, not just metrics. Every metric has a "do something about it" link. ### 6.2 Where It Lives **Not a new page.** It replaces the current Overview tab content on vault detail. The Overview tab already answers "what is my agent up to?" — the dashboard evolves it to answer "how well is my agent's knowledge being refined?" URL: `/vaults/[slug]` (same as today, Overview tab is default). ### 6.3 Dashboard Sections ``` Vault: my-coding-agent [Overview] Memories | ... --------------------------------------------------------------------------- DISTILLATION HEALTH Score: 72/100 Conversations ----[=====>]---- Memories ----[===>]---- Knowledge ----[=>]---- Skills 142 347 12 3 87% extracted 3% structured 25% crystallized --------------------------------------------------------------------------- RECOMMENDED ACTIONS 3 suggestions [!] 47 memories tagged "preferences" have never been consolidated These memories share themes. A knowledge document would be more useful to your agent than 47 individual memories. [Consolidate now ->] [!] "Deploy Process" (Knowledge) describes a 6-step procedure Procedural knowledge is more effective as an executable skill. [Promote to Skill ->] [i] 23 conversations from last week have no extracted memories Your agent may have missed observations worth preserving. [Review conversations ->] (V1.2+) --------------------------------------------------------------------------- VAULT STATS Level Count Avg Density Total Tokens Coverage Conversations 142 1x ~284,000 — Memories 347 8x ~17,350 87% of conversations Knowledge 12 18x ~3,600 3% of memories Skills 3 27x ~900 25% of knowledge Compression ratio: 284,000 tokens of experience -> 900 tokens of skills (315:1) --------------------------------------------------------------------------- RECENT DISTILLATION ACTIVITY Mar 18 Consolidated 23 memories -> "Coding Standards" (Knowledge) Mar 15 Promoted "Deploy Process" -> deploy-weekly-report (Skill) Mar 12 Extracted 4 memories from conversation #89 Mar 10 Consolidated 11 memories -> "API Conventions" (Knowledge) ``` ### 6.4 Metrics Defined | Metric | Definition | How Calculated | |--------|-----------|----------------| | **Extraction rate** | % of conversations that produced at least one memory | `conversations_with_memories / total_conversations * 100` | | **Structuring rate** | % of memories that have been consolidated into knowledge docs | `memories_with_consolidated_tag / total_active_memories * 100` | | **Crystallization rate** | % of procedural knowledge docs promoted to skills | `knowledge_with_promoted_tag / procedural_knowledge_count * 100` | | **Distillation score** | Composite health score (0-100) | Weighted: extraction 40% + structuring 40% + crystallization 20% | | **Compression ratio** | Total conversation tokens / total skill tokens | Rough estimate based on average token counts per level | | **Knowledge density** (per level) | Actionable info per token, relative to conversations | Derived from the density multiplier table (Section 2.3) | ### 6.5 Suggestion Engine Suggestions are generated by simple heuristics, not AI calls (to avoid LLM dependency on page load): 1. **"Consolidate memories"** triggers when: 10+ memories share a tag, AND no knowledge doc with that tag exists. 2. **"Promote to skill"** triggers when: a knowledge doc has type "Procedure" AND no skill references it. 3. **"Review conversations"** triggers when: conversations exist with no linked memories (by date proximity or explicit link). Available V1.2+. 4. **"Memory hygiene needed"** triggers when: the Memory Hygiene feature (ai-features-design.md Feature 3) detects 5+ issues. ### 6.6 Design Principles for the Dashboard - **Kenya Hara:** The pipeline visualization should feel like a natural flow, not a factory diagram. The width of each pipeline segment reflects relative volume. Wider at conversations, narrowing toward skills. Like a river filtering through increasingly fine soil. - **Allen Zhang:** No configuration. No settings. The dashboard reads vault state and shows what matters. The user should feel like the product understands their vault, not like they are operating an analytics tool. - **Jobs:** The distillation score is the only number that matters. If it is high, the vault is healthy. If it is low, the suggestions explain why. One number, clear actions. - **Shannon:** The compression ratio is the information-theoretic headline. 315:1 means the vault has extracted 315x more value per token than raw conversation storage. That is the number that proves distillation works. --- ## Appendix: What This Document Does NOT Cover - **Agent-side extraction improvements.** How agents write better memories is outside KnowHub console scope. KnowHub observes and refines what agents produce; it does not control agent behavior. - **Semantic search.** Still deferred to V2. Consolidation does not require embeddings — it uses selected memories as direct input. - **Cross-vault distillation.** Shared spaces may eventually allow consolidating memories from multiple vaults into shared knowledge. Deferred. - **Automated distillation.** All transitions are human-initiated in V1.1/V1.2. Fully automated "agent experience -> skill" pipelines are a V2+ exploration. --- *KnowHub is not a filing cabinet. It is a distillery. Raw experience goes in. Refined expertise comes out. The human decides what is worth keeping at every step.*