# KnowHub AI Features — Implementation Architecture

**Date:** 2026-03-19 | **Status:** Implementation Spec

---

## 1. Architecture Decision: One Unified Route

**Choice: Option C — One route (`POST /api/ai/invoke`) with prompt templates selected by `action`.**

Rationale (Carmack wins): The LLM is a function — it takes a prompt, returns text. The only things that vary per feature are the prompt and the response schema. One route means one place to read config headers, one place to handle provider differences, one error path. Five routes would duplicate 90% of the code.

The `action` parameter selects a prompt template. The route constructs the messages, calls the provider, parses JSON from the response, returns it.

---

## 2. The AI Route — `frontend/src/app/api/ai/invoke/route.ts`

### Request Format

```typescript
// POST /api/ai/invoke
interface AIInvokeRequest {
  action: 'extract-metadata' | 'analyze-skill' | 'check-memories' | 'check-staleness';
  payload: Record<string, unknown>; // action-specific data
}
```

Note: `classify-content` is merged into `extract-metadata` (one LLM call, not two — Allen Zhang's rule).

### Headers (same pattern as auth proxy)

```
X-KnowHub-AI-Provider: "openai" | "anthropic" | "custom"
X-KnowHub-AI-Key: <api-key>
X-KnowHub-AI-Model: "gpt-4o-mini" | "claude-haiku-3" | <custom>
X-KnowHub-AI-BaseURL: "https://api.openai.com/v1" | <custom>
```

### Provider Dispatch (inside route.ts)

```typescript
// Normalize all providers to OpenAI-compatible chat completions format.
// Anthropic uses /v1/messages with a different shape — map it.
// Custom providers must be OpenAI-compatible (most are).

async function callLLM(provider, key, model, baseUrl, messages, maxTokens) {
  if (provider === 'anthropic') {
    // POST {baseUrl}/v1/messages — map messages to Anthropic format
    // system prompt goes in top-level `system` field, not in messages array
    // Header: x-api-key (not Authorization: Bearer)
    return callAnthropic(key, model, baseUrl, messages, maxTokens);
  }
  // OpenAI and Custom: POST {baseUrl}/chat/completions
  // Header: Authorization: Bearer {key}
  return callOpenAI(key, model, baseUrl, messages, maxTokens);
}
```

### Response Format (all actions)

```typescript
interface AIInvokeResponse {
  ok: boolean;
  data: Record<string, unknown>; // action-specific parsed JSON
  error?: string;                // only if ok === false
  tokens_used?: number;          // for usage tracking
}
```

### Error Handling Within the Route

1. Missing/invalid headers: return `{ ok: false, error: "ai_not_configured" }` (400)
2. LLM returns non-JSON: attempt to extract JSON from markdown code fences (`/```json?\n([\s\S]*?)\n```/`), then try raw parse. If both fail: return `{ ok: false, error: "invalid_response" }` (502)
3. LLM timeout (>15s): AbortController with 15s signal. Return `{ ok: false, error: "timeout" }` (504)
4. LLM 401/403: return `{ ok: false, error: "invalid_key" }` (401)
5. LLM 429: return `{ ok: false, error: "rate_limited" }` (429)
6. Token limit exceeded (LLM returns 400 with context_length): return `{ ok: false, error: "content_too_long" }` (413)

---

## 3. Prompt Templates

### Feature 1+4: Smart Upload + Skill Identifier (combined)

```
Action: "extract-metadata"
Payload: { content: string, filename: string }
Max tokens: 300

System prompt:
"Extract metadata from a file uploaded to a knowledge management system. Respond with ONLY a JSON object, no other text."

User prompt:
"Filename: {filename}

Content (first 3000 chars):
{content}

Return JSON:
{
  "title": "concise title",
  "description": "one sentence describing what this file contains, written so an AI agent can judge relevance",
  "tags": ["2-5 lowercase tags"],
  "classification": "skill" or "knowledge",
  "classification_confidence": "high" or "medium" or "low",
  "classification_reasoning": "one sentence why",
  "knowledge_subtype": "reference" or "procedure" or "template" (only if classification is knowledge)
}"

Expected response schema:
{
  "title": string,
  "description": string,
  "tags": string[],
  "classification": "skill" | "knowledge",
  "classification_confidence": "high" | "medium" | "low",
  "classification_reasoning": string,
  "knowledge_subtype"?: "reference" | "procedure" | "template"
}
```

### Feature 2: Skill Analyzer

```
Action: "analyze-skill"
Payload: { skillMd: string }
Max tokens: 500

System prompt:
"You evaluate skill definitions for AI agents. Score quality and suggest fixes. Respond with ONLY a JSON object, no other text."

User prompt:
"Analyze this skill definition:

{skillMd}

Score 1-5 on each dimension. For any score below 4, provide one concrete fix (a specific rewrite or addition, not generic advice).

Return JSON:
{
  "scores": {
    "clarity": 1-5,
    "completeness": 1-5,
    "specificity": 1-5,
    "efficiency": 1-5
  },
  "fixes": [
    { "dimension": "clarity|completeness|specificity|efficiency", "problem": "what is wrong", "suggestion": "specific fix text" }
  ]
}"

Expected response schema:
{
  "scores": { "clarity": number, "completeness": number, "specificity": number, "efficiency": number },
  "fixes": { "dimension": string, "problem": string, "suggestion": string }[]
}
```

### Feature 3: Memory Hygiene

```
Action: "check-memories"
Payload: { memories: { id: string, title: string, content: string }[] }
Max tokens: 800

System prompt:
"You analyze a list of AI agent memories to find duplicates and contradictions. Respond with ONLY a JSON object, no other text."

User prompt:
"Memories:
{memories as numbered list: "1. [id] Title: {title}\n{content (first 200 chars)}\n---"}

Identify:
- Groups of memories that say the same thing (duplicates)
- Pairs of memories that contradict each other

Return JSON:
{
  "duplicates": [
    { "ids": ["id1","id2"], "explanation": "one sentence" }
  ],
  "contradictions": [
    { "ids": ["id1","id2"], "explanation": "one sentence" }
  ]
}"

Expected response schema:
{
  "duplicates": { "ids": string[], "explanation": string }[],
  "contradictions": { "ids": string[], "explanation": string }[]
}
```

### Feature 5: Staleness Check

```
Action: "check-staleness"
Payload: { documents: { id: string, filename: string, content: string, lastModified: string }[] }
Max tokens: 600

System prompt:
"You check knowledge documents for outdated content. Respond with ONLY a JSON object, no other text."

User prompt:
"Today's date: {current date}

Documents:
{documents as numbered list: "1. [id] {filename} (last modified: {lastModified})\n{content (first 500 chars)}\n---"}

For each document, determine if it may be outdated based on: date references older than 6 months, version numbers that may be superseded, references to tools/APIs that change frequently.

Return JSON:
{
  "results": [
    { "id": "doc-id", "isStale": true|false, "reasons": ["specific reason"], "summary": "one sentence" }
  ]
}"

Expected response schema:
{
  "results": { "id": string, "isStale": boolean, "reasons": string[], "summary": string }[]
}
```

---

## 4. Client-Side Helpers — `frontend/src/lib/ai.ts`

```typescript
// --- Config ---

interface AIConfig {
  provider: 'openai' | 'anthropic' | 'custom';
  apiKey: string;
  model: string;
  baseUrl: string;
  enabledFeatures: Record<string, boolean>;
  tokenBudget: number;
}

const AI_CONFIG_KEY = 'knowhub_ai_config';

export function getAIConfig(): AIConfig | null {
  const raw = localStorage.getItem(AI_CONFIG_KEY);
  if (!raw) return null;
  try { return JSON.parse(raw); } catch { return null; }
}

export function saveAIConfig(config: AIConfig): void {
  localStorage.setItem(AI_CONFIG_KEY, JSON.stringify(config));
}

export function isFeatureEnabled(feature: string): boolean {
  const config = getAIConfig();
  return !!(config?.apiKey && config.enabledFeatures?.[feature] !== false);
}

// --- Invoke (single entry point) ---

async function invokeAI<T>(action: string, payload: Record<string, unknown>): Promise<T | null> {
  const config = getAIConfig();
  if (!config?.apiKey) return null;

  const res = await fetch('/api/ai/invoke', {
    method: 'POST',
    headers: {
      'Content-Type': 'application/json',
      'X-KnowHub-AI-Provider': config.provider,
      'X-KnowHub-AI-Key': config.apiKey,
      'X-KnowHub-AI-Model': config.model,
      'X-KnowHub-AI-BaseURL': config.baseUrl,
    },
    body: JSON.stringify({ action, payload }),
  });

  if (!res.ok) return null;
  const json = await res.json();
  return json.ok ? json.data as T : null;
}

// --- Feature functions (what components import) ---

export async function extractMetadata(content: string, filename: string) {
  return invokeAI<{
    title: string; description: string; tags: string[];
    classification: 'skill' | 'knowledge';
    classification_confidence: 'high' | 'medium' | 'low';
    classification_reasoning: string;
    knowledge_subtype?: string;
  }>('extract-metadata', { content, filename });
}

export async function analyzeSkill(skillMd: string) {
  const cacheKey = `knowhub_ai_skill_${hashString(skillMd)}`;
  const cached = sessionStorage.getItem(cacheKey);
  if (cached) return JSON.parse(cached);

  const result = await invokeAI<{
    scores: { clarity: number; completeness: number; specificity: number; efficiency: number };
    fixes: { dimension: string; problem: string; suggestion: string }[];
  }>('analyze-skill', { skillMd });

  if (result) sessionStorage.setItem(cacheKey, JSON.stringify(result));
  return result;
}

export async function checkMemoryHygiene(memories: { id: string; title: string; content: string }[]) {
  const cacheKey = 'knowhub_ai_memory_hygiene';
  const cacheEntry = sessionStorage.getItem(cacheKey);
  if (cacheEntry) {
    const { data, timestamp } = JSON.parse(cacheEntry);
    if (Date.now() - timestamp < 3600_000) return data; // 1 hour cache
  }

  const result = await invokeAI<{
    duplicates: { ids: string[]; explanation: string }[];
    contradictions: { ids: string[]; explanation: string }[];
  }>('check-memories', { memories });

  if (result) sessionStorage.setItem(cacheKey, JSON.stringify({ data: result, timestamp: Date.now() }));
  return result;
}

export async function checkStaleness(documents: { id: string; filename: string; content: string; lastModified: string }[]) {
  const cacheKey = 'knowhub_ai_staleness';
  const cacheEntry = localStorage.getItem(cacheKey);
  if (cacheEntry) {
    const { data, timestamp } = JSON.parse(cacheEntry);
    if (Date.now() - timestamp < 86_400_000) return data; // 24 hour cache
  }

  const result = await invokeAI<{
    results: { id: string; isStale: boolean; reasons: string[]; summary: string }[];
  }>('check-staleness', { documents });

  if (result) localStorage.setItem(cacheKey, JSON.stringify({ data: result, timestamp: Date.now() }));
  return result;
}

// --- Utility ---

function hashString(str: string): string {
  let hash = 0;
  for (let i = 0; i < str.length; i++) {
    hash = ((hash << 5) - hash + str.charCodeAt(i)) | 0;
  }
  return hash.toString(36);
}
```

---

## 5. Caching Strategy

| Feature | Store | Key | TTL | Bust on |
|---------|-------|-----|-----|---------|
| Smart Upload + Classify | None | — | — | Each file unique |
| Skill Analyzer | sessionStorage | `knowhub_ai_skill_{contentHash}` | Session | New content hash (edit) |
| Memory Hygiene | sessionStorage | `knowhub_ai_memory_hygiene` | 1 hour | TTL expiry |
| Staleness Check | localStorage | `knowhub_ai_staleness` | 24 hours | TTL expiry |

Why client-side storage, not server-side: The Next.js route is stateless (no database). The backend is Gitea (not ours). localStorage/sessionStorage is the only state we control, and it already stores auth config. This is consistent with the existing architecture.

Cache invalidation for Skill Analyzer: The key includes a hash of the skill content. If the user edits the skill, the hash changes, and the next analysis call misses cache. Old entries are in sessionStorage and die with the tab.

---

## 6. AI Config Panel — `frontend/src/components/ai-config-panel.tsx`

Component spec:

```
AIConfigPanel (rendered in Settings page under an "AI" tab)
├── ProviderSection
│   ├── Provider selector: three buttons (OpenAI / Anthropic / Custom)
│   ├── API Key: password input + "Test Connection" button
│   ├── Model: text input (pre-filled with default per provider)
│   │   - OpenAI default: "gpt-4o-mini"
│   │   - Anthropic default: "claude-haiku-3"
│   │   - Custom: empty
│   └── Base URL: text input (pre-filled, editable only for Custom)
│       - OpenAI default: "https://api.openai.com/v1"
│       - Anthropic default: "https://api.anthropic.com"
├── FeaturesSection
│   └── Five toggles, one per feature, all default ON
└── Save button → writes to localStorage(AI_CONFIG_KEY)
```

"Test Connection" sends: `POST /api/ai/invoke` with action `"test"` and empty payload. The route sends a minimal prompt ("Say ok") and returns success/failure. The UI shows a green dot or red error inline.

Usage tracking: Each successful `invokeAI` call increments counters in localStorage (`knowhub_ai_usage`): `{ requests: number, tokens: number, month: string }`. Displayed read-only at bottom of config panel. Reset when month changes.

---

## 7. Files to Create / Modify

### New Files (3)

| File | Purpose | Lines (est.) |
|------|---------|-------------|
| `frontend/src/app/api/ai/invoke/route.ts` | Unified AI route: reads headers, selects prompt template, calls LLM provider, parses JSON response, returns data | ~200 |
| `frontend/src/lib/ai.ts` | Client helpers: `getAIConfig`, `invokeAI`, `extractMetadata`, `analyzeSkill`, `checkMemoryHygiene`, `checkStaleness`, `isFeatureEnabled` | ~120 |
| `frontend/src/components/ai-config-panel.tsx` | Settings UI: provider config, feature toggles, test connection, usage display | ~180 |

### Modified Files (6)

| File | Change |
|------|--------|
| `frontend/src/components/file-upload-zone.tsx` | After file read, call `extractMetadata()` if enabled. Pre-fill metadata fields with result. Show classification badge with confidence. |
| `frontend/src/components/skill-detail.tsx` | Add "Analyze" button (hidden if `!isFeatureEnabled('analyze')`). On click, call `analyzeSkill()`, render score bars and fix suggestions in a collapsible panel below the header. |
| `frontend/src/components/memory-card.tsx` (or parent list) | On Memories tab mount, if 10+ memories and feature enabled, call `checkMemoryHygiene()`. Show banner with issue count. Wire "Review" to open a drawer. |
| `frontend/src/components/document-card.tsx` | Accept optional `stalenessInfo` prop. If `isStale`, render a muted warning badge on the card. |
| Settings page (wherever tabs are rendered) | Add "AI" tab that renders `<AIConfigPanel />`. |
| `frontend/src/lib/types.ts` | Add `AIConfig` and related response types. |

### Not Created (by design)

- No separate routes per feature (one route handles all).
- No server-side cache layer (client storage is sufficient).
- No AI-specific middleware (the route handles its own auth).
- No streaming — all responses are small JSON. Streaming adds complexity with no benefit here.

---

## Summary of Key Decisions

1. **One route, one function call.** `POST /api/ai/invoke` with an `action` field. No route proliferation.
2. **Smart Upload + Skill Identifier = one LLM call.** Classification is just another field in the metadata extraction prompt.
3. **Batching for Memory Hygiene and Staleness.** All memories in one call. All knowledge docs in one call. Never per-item.
4. **Client-side caching with TTL.** sessionStorage for session-scoped data, localStorage for persistent data. No server state needed.
5. **Graceful degradation everywhere.** Every feature returns `null` on failure. Components check `null` and hide AI UI. No error modals for background features.
6. **JSON extraction with fallback.** Try direct parse, then regex for fenced code blocks. Both GPT-4o-mini and Claude Haiku reliably produce JSON when instructed.
7. **Content truncation in prompts.** File content capped at 3000 chars for metadata, 500 chars per doc for staleness batches, 200 chars per memory for hygiene batches. Keeps token costs predictable.