---
title: Skill Reliability Research
---

## Summary

Consolidated findings from web articles and GitHub repositories on approaches, patterns, and best practices for making AI agent skills more reliable. Research conducted 2026-02-14.

## Top-Tier Patterns (High Impact, Well-Validated)

| Pattern | Source | How It Applies |
|---------|--------|----------------|
| Invest in descriptions | Anthropic/OpenAI docs, all frameworks | `description:` field is the #1 reliability lever. Include when to use, when NOT to use, examples, and boundaries. |
| Progressive disclosure | Agent Skills spec, VoltAgent, Context Engineering | Keep SKILL.md metadata under ~100 tokens; body under 500 lines. Load companion files lazily. |
| RFC 2119 constraints | Strands Agent SOP (736 stars) | Use MUST/SHOULD/MAY in skill instructions to remove ambiguity. |
| Verification-before-completion | obra/superpowers (480 stars), Trail of Bits (2,623 stars) | Dedicated step/skill that forces the agent to verify results before declaring done. |
| Ask-before-implementing | Trail of Bits `ask-questions-if-underspecified` | Skill that forces clarification on ambiguous inputs before execution. |
| Negative examples | Anthropic prompt guide | Show what NOT to do. Prevents common misuse patterns. |
| Scoped tool declarations | Agent Skills spec quality standards | Request only needed tools; never blanket `"tools": ["*"]`. |

## Mid-Tier Patterns (Valuable for Complex Skills)

| Pattern | Source | Description |
|---------|--------|-------------|
| Schema-based output validation | Instructor (12K stars), Guardrails (6K stars) | Validate outputs against schemas; retry with error feedback on failure. |
| Multi-agent verification | SkillForge Phase 4 (508 stars) | Multiple specialized agents must unanimously approve output. |
| Gate patterns | Clarity Gate | Documents must pass quality checks before entering knowledge base. |
| Eval/test suites | Promptfoo (10K stars), DeepEval (13K stars) | Golden dataset + assertion-based tests. Treat prompts as code, test in CI. |
| Retry with error feedback | Instructor, LiteLLM, Guardrails | On failure, feed validation error back to model for self-correction. |
| Context degradation awareness | Context Engineering skills | Recognize lost-in-middle, context poisoning, attention scarcity. |
| Artifact persistence | Strands SOP, obra/superpowers | Write plans/scratchpads to filesystem for state continuity across sessions. |

## Key Repos

| Repo | Stars | Why It Matters |
|------|-------|----------------|
| VoltAgent/awesome-agent-skills | 6,962 | Defines skill quality standards; largest curated collection |
| trailofbits/skills | 2,623 | Security-focused; `ask-questions-if-underspecified` + `verification-before-completion` patterns |
| obra/superpowers | 480 | Composable dev workflow skills; mandatory verification, TDD enforcement |
| tripleyak/SkillForge | 508 | 4-phase skill generation with multi-agent approval gate |
| strands-agents/agent-sop | 736 | RFC 2119 constraints in markdown SOPs; multi-modal distribution |
| anthropics/skills | Official | Canonical skill format; spec at agentskills.io |
| promptfoo/promptfoo | 10,445 | Declarative skill testing in CI/CD |
| PatrickJS/awesome-cursorrules | 37,831 | Massive .cursorrules collection for constraining AI behavior |
| stanfordnlp/dspy | 32,179 | "Programming not prompting" -- compiles declarative LLM calls into optimized pipelines |
| 567-labs/instructor | 12,366 | Structured outputs with Pydantic; automatic retry on validation failure |
| guardrails-ai/guardrails | 6,398 | Output validators for quality, format, toxicity, PII |
| langfuse/langfuse | 21,897 | Open source LLM observability for debugging skill failures |
| disler/claude-code-hooks-mastery | 3,020 | Claude Code hooks for runtime policy enforcement |

## Detailed Findings by Category

### 1. Tool/Skill Definition Best Practices

- Clear verb-noun tool names (e.g., `get_weather`, `search_database`)
- Detailed descriptions: what it does, when to use, when NOT to use, expected input format, example usage
- Precise parameter schemas: `enum` for constrained values, `description` on every parameter
- Minimize tool count: fewer tools reduces misselection risk
- Provide 1-2 example invocations directly in descriptions

### 2. Prompt Engineering for Reliability

- Place tool usage instructions in system prompt with explicit sections
- Chain-of-thought before tool calls: reason about which tool before calling
- Few-shot examples: 2-3 correct usage patterns in prompt
- Negative examples: show what NOT to do
- Output format anchoring: specify exact expected format
- Role/persona: define agent role clearly

### 3. Structured Output and Validation

- Constrained decoding / structured outputs guarantee schema compliance
- Pydantic/Zod schemas for response models
- Retry with error feedback: send validation error back to model for correction
- Enum fields over free text reduces hallucination
- Nested schemas: break complex outputs into nested objects

### 4. Error Handling and Retry

- Graceful degradation: return structured errors to model, not crashes
- Exponential backoff with jitter for transient failures
- Max retry limits (typically 3) to prevent infinite loops
- Error classification: retryable (timeout, rate limit) vs non-retryable (invalid input)
- Self-healing prompts: feed error back, ask model to correct invocation
- Idempotent tool design where possible

### 5. Context Window Management

- Summarization chains for long conversations
- RAG over stuffing everything into context
- Sliding window with anchor messages (system prompt, key decisions)
- Tool result truncation: summarize large outputs
- Progressive disclosure: high-level first, details on request
- Context budgeting: ~20% system, ~60% conversation, ~20% generation headroom

### 6. Evaluation and Testing

- Assertion-based evals: deterministic assertions on tool call outputs
- LLM-as-judge for subjective quality evaluation
- Golden dataset testing as regression tests
- A/B testing of prompts: compare eval results across versions
- Key metrics: tool selection accuracy, parameter accuracy, task completion rate
- Trace logging: log every tool call, params, result, reasoning
- Continuous evaluation in CI/CD on every prompt change

### 7. Guardrails and Safety

- Input validation before passing to tools
- Output validation against expected schemas
- Rate limiting per-user and per-tool
- Sandboxing for tool execution
- Human-in-the-loop for high-stakes actions
- Circuit breakers for repeatedly failing tools
- Audit logging for post-incident analysis

### 8. Agent Orchestration

- Prompt chaining over autonomous agents for deterministic workflows
- Routing patterns: classifier/router to specialized sub-agents
- Orchestrator-workers: break tasks, delegate to focused workers
- State machines for explicit workflow transitions
- Checkpointing at key milestones
- Plan-then-execute: generate plan, validate, then execute step by step
- Anthropic guidance: "Start with prompt chains, add complexity only when needed"

### 9. Determinism and Reproducibility

- Temperature 0 for deterministic tool-calling
- Seed parameter for reproducible outputs
- Deterministic (unambiguous) tool descriptions
- Caching identical tool calls
- Version pinning model versions in production
- Snapshot testing for tool call sequences

### 10. Real-World Patterns from GitHub

- **Progressive disclosure**: load only skill names/descriptions at startup, full content on activation (Agent Skills spec, VoltAgent)
- **RFC 2119 constraints**: MUST/SHOULD/MAY keywords make requirements unambiguous (Strands SOP)
- **Multi-agent verification**: panel of specialized agents must unanimously approve (SkillForge)
- **Verification-before-completion**: dedicated skill forces verification before done (superpowers, Trail of Bits)
- **Ask-before-implementing**: force clarification on ambiguous inputs (Trail of Bits)
- **Gate patterns**: documents must pass quality checks before use (Clarity Gate)
- **Scoped tool declarations**: request only needed tools (Agent Skills spec)
- **Retry with feedback**: feed validation error back to LLM for correction (Instructor, Guardrails)
- **Context degradation awareness**: recognize lost-in-middle, poisoning, distraction (Context Engineering)
- **Artifact persistence**: write plans/scratchpads to filesystem for continuity (Strands SOP, superpowers)
- **Trophy case**: document real bugs found to validate skill effectiveness (Trail of Bits)

## Actionable Recommendations (Priority Order)

1. **Enrich `description:` fields** -- add when-to-use, when-NOT-to-use, 1-2 examples
2. **Add negative examples** -- show common failure modes and how to avoid them
3. **Use RFC 2119 keywords** (MUST/SHOULD/MAY) instead of soft language
4. **Add verification steps** -- skills should end with explicit "verify the result" instructions
5. **Scope tools tightly** -- agents should declare only the tools they need
6. **Progressive disclosure** -- keep skill entry points lean; push detail into `components/`
7. **Consider eval testing** -- promptfoo-style assertions for critical skill behaviors
8. **Add clarification gates** -- instruct agent to ask before acting on ambiguous input
9. **Document failure modes** -- "Common Pitfalls" section per skill prevents repeated mistakes
10. **Trophy case** -- track real successes/failures to validate skill effectiveness over time

## Caveats

- foo's web research was compiled from training knowledge (WebFetch was blocked in environment)
- bar's GitHub research used live `gh search` and WebSearch/WebFetch; star counts are as of 2026-02-14
- Some repos may have been renamed or archived since research date