# CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

## Architecture

This is `clara_fts`, a full-text search library built on top of Tantivy. It provides a simplified interface for indexing and searching text documents with support for multiple tokenizers.

### Core Components

**Index Management:**

- `IndexWriter` - Creates and writes to search indexes using Tantivy's SingleSegmentIndexWriter
- `IndexReader` - Reads from existing indexes, supports memory-mapped files and in-memory buffers
- Both support FFI-compatible interfaces with primitive types

**Directory System:**

- `TrackedDirectory` - Wrapper that tracks file operations for debugging/monitoring

**Tokenizers:**

- `StandardV1Tokenizer` - Basic text tokenization
- `MultilingualV1Tokenizer` - Supports Chinese segmentation via charabia library
- Tokenizer registry managed through lazy_static TOKENIZERS

**Query System:**

- Located in `src/query/` (new module)
- Provides query parsing and execution capabilities

**Compatibility:**

- `tantivy_compat` module handles compatibility with different Tantivy versions
- Includes field norm encoding utilities

### Key Design Patterns

- Uses a fixed field schema with single "body" field (FIELD_BODY constant)
- Protobuf code generation for directory serialization (build.rs compiles proto.proto)
- FFI-friendly APIs using primitive types for cross-language integration
- Memory-mapped file support for efficient large index handling

### Testing

Tests are embedded throughout the codebase using `#[cfg(test)]` and `#[test]` attributes. The library uses the standard Rust test framework with some benchmark tests using the nightly `test` crate feature.

## Development Guidelines

These principles guide all development work on clara_fts, regardless of which component you're working on.

### Testing Philosophy

- **Tests as documentation**: Tests serve as living documentation of expected behavior and API usage patterns
- **Maintain coverage**: All changes must preserve existing test coverage and add tests for new functionality
- **Ergonomic test utilities**: Test helper functions should return clean types and handle errors internally (use `.unwrap()` for fail-fast behavior)

### API Design Principles

- **Simplicity over convenience**: Prefer direct, simple APIs over multiple wrapper functions or redundant helper methods
- **Avoid redundancy**: Don't create multiple ways to do the same thing; consolidate to a single, well-designed approach
- **Error handling strategy**: Match error handling to context - `Result<T>` for library code, `.unwrap()` for test utilities

### Protobuf Integration (tipb)

Working with the tipb crate and protobuf-generated code requires understanding several key patterns:

- **Generated code location**: tipb protobuf definitions are generated at build time in `target/debug/build/tipb-*/out/protos/`
- **Method patterns**: Use `mut_field().push()` instead of creating `RepeatedField` manually (e.g., `info.mut_columns().push(column_info)`)
- **Workspace dependencies**: tipb is a workspace dependency, not directly listed in clara_fts Cargo.toml
- **Field evolution**: API methods may change (e.g., `set_column_id()` -> `set_columns()` with ColumnInfo), check generated code
- **Exploration technique**: Use `find /path -name "*.rs" | xargs grep -l "TypeName"` to locate generated struct definitions and available methods

**Common tipb patterns in clara_fts:**

- `FtsQueryInfo` construction with query text, tokenizer, and column information
- Using `PlainFtsQueryInfo` as a convenient builder pattern in tests
- Converting between plain structs and protobuf messages via `as_query_info()` methods