# How Filtering Makes Vector Search Practical in a Hybrid Search Engine Vector search becomes far more useful when it works inside real business constraints rather than across an entire embedding corpus. Production workloads rarely search "everything." They search within a tenant, a permission boundary, a time window, a catalog state, a language, or a product slice. Once those constraints matter, vector retrieval is no longer only a nearest-neighbor problem. It becomes an execution-planning problem: the engine has to decide which retrieval path is both correct and efficient for the filtered search space. That is the core idea behind filtering in Search Fabric. We do not treat filtering as a cosmetic cleanup step after vector retrieval. We treat it as a first-class input into query planning. ## Why filtering changes the economics of vector search Without filters, approximate nearest neighbor (ANN) search is usually the right default. It avoids scanning the full vector space and provides a good latency-versus-recall trade-off at large scale. Once filters are added, that trade-off changes: - If the filter is highly selective, the surviving candidate set may already be small enough that an exact distance pass is cheaper and more reliable than traversing an ANN graph. - If the filter is broad, ANN is still valuable, but only if the engine can apply the filter early enough to avoid wasting work on documents that will be rejected later. - If the filter is enforced only at the end, the system may underfill `top_k`, spend budget on ineligible documents, and make recall depend on heuristic overfetch instead of real selectivity. This is why filtered vector search cannot be reduced to "ANN plus a WHERE clause." | Retrieval pattern | Works well when | Main limitation | | --- | --- | --- | | Exact distance over full corpus | Collections are small | Too expensive at scale | | ANN then post-filter | Filters are soft and overfetch is cheap | Can waste work and underfill results | | Metadata-aware filtered plan | Queries mix vectors with hard business constraints | Requires a real planner and cross-index cooperation | ## Why post-filtering cannot be the default Post-filtering is simple, and it has its place. For loose filters and generous overfetch budgets, it can be good enough. But it is a poor default for hard constraints such as tenant isolation, ACL boundaries, availability state, or freshness windows. In those cases, the filter is not advisory. It is the rule that determines whether a document is eligible at all. If the engine retrieves vector candidates first and filters later, three problems appear quickly: - the ANN stage spends most of its budget exploring documents that can never be returned; - too many candidates can disappear after ranking, leaving the final result set underfilled; - result quality becomes sensitive to heuristic tuning rather than to the real size of the filtered population. In a production hybrid engine, hard filters have to shape the retrieval plan itself. ## Filtering starts in the metadata path Search Fabric handles filtered vector search by starting in the metadata path, not inside the vector index alone. The planner, called the `Query Broker`, pins a shared snapshot timestamp (`read_ts`) for the entire query. It then asks the `Metadata Search Reader` to evaluate structured predicates and text clauses under that same snapshot. This stage: - applies MVCC visibility masks; - merges query-visible near-real-time deltas; - estimates how many documents survive the filter; - and returns either an explicit candidate set or a compact filter representation for the vector path. This is a strategic choice. The metadata path already has the best signal for filter selectivity, so the vector path should not guess. It should receive a concrete plan. On the metadata side, predicate evaluation follows a clear order: - inverted-index set algebra first, with intersections for `AND` logic and unions for `OR` logic; - auxiliary index evaluation when inverted coverage is incomplete; - bitmap or Bloom runtime filters when they are the cheapest transfer format; - fast-field filtering or bounded scans only as the fallback path. That separation lets the hybrid engine use each subsystem for what it is best at: metadata search for structured selectivity, vector search for semantic similarity, and the broker for choosing the right physical plan. ## Two execution paths instead of one Once the metadata side measures selectivity under the same snapshot, the system can choose between two vector execution modes. ### Path A: bounded exact distance for a small surviving set When the filter is highly selective, the metadata side returns an explicit primary-key set. At that point, the `Semantic Search Reader` can skip ANN graph traversal and run a bounded exact distance pass over that filtered set, using compressed-vector evaluation rather than treating the entire corpus as the search space. This path is valuable because: - recall becomes deterministic for the surviving set; - the engine avoids masked-graph pathologies caused by heavily filtered ANN neighborhoods; - and it stops paying ANN overhead after the filter has already done most of the pruning. The crossover point is workload-calibrated rather than universal, but the design assumes a genuine "small candidate set" regime instead of forcing every filtered query through ANN. ### Path B: masked ANN traversal for a broad surviving set When the filter is broader, returning a full key list becomes wasteful. In that case, the metadata side sends a compact filter payload, typically an aligned bitmap or a Bloom filter, and the `Semantic Search Reader` runs masked ANN traversal against the incremental vector graph. This keeps approximate retrieval where approximation still helps, but it is no longer blind ANN. The vector path evaluates only candidates that pass the transferred filter payload. If a Bloom filter is used, the engine performs an exact final filter check before returning results so Bloom false positives never leak into the response. When metadata and vector segments share aligned internal ordinals, bitmap transfer becomes especially efficient because the system can move filter state across modalities without translating a large set of document IDs. The decision flow looks like this: ```text hard filter + shared snapshot | v metadata evaluation and selectivity estimate | +-----+-----+ | | small set broad set | | PK set bitmap/Bloom | | exact pass masked ANN +-----+-----+ | final authoritative filter | fused response ``` ## Correctness depends on snapshot visibility, not only on predicates In a hybrid engine, "correct filtering" means more than respecting the user-visible predicate. The system also has to respect: - the query snapshot (`read_ts`); - tombstones and version visibility; - near-real-time updates that are committed but not yet flushed into immutable segments; - and final fusion across metadata and semantic candidate streams. That is why filtering is enforced in multiple places, each with a different role: - metadata search applies external visibility bitmaps and merges visible metadata deltas; - semantic search applies snapshot checks, tombstone filtering, and visible vector deltas; - the broker fuses metadata and semantic candidates, deduplicates by document identity, and performs a final authoritative scalar-filter validation before emission and pagination. The pre-filter payload sent to the vector path is an optimization. It is not the source of truth. This distinction is important because it lets the engine change physical execution plans without changing logical answers. ## One authoritative filter, plus local optimization hints The query contract keeps a clean boundary between correctness and optimization. There is one global hard filter for the whole query, and there may also be clause-local vector filters or strategy hints that help the planner pick a better execution path. Those local hints can influence whether the system chooses masked ANN, bounded exact search, iterative filtering, or another physical strategy. They cannot expand eligibility. Before ANN traversal, the planner combines the global hard filter with any vector-local pruning rule, or dispatches an equivalent joint evaluation plan. After candidate fusion and deduplication, the engine checks the authoritative global filter again before returning results or continuing pagination. This is what allows the platform to expose execution preferences without turning them into correctness loopholes. Strategy selection remains flexible, while the final answer remains stable. ## Why this matters to customers Treating filtering as a planning problem produces three practical benefits for customer-facing search workloads: - better result quality under hard business constraints, because the system does not rely on blind ANN plus aggressive overfetch; - more predictable latency, because the planner can switch to the cheapest correct path for the filtered population; - a cleaner hybrid model, because metadata, text, and vector retrieval all run under the same snapshot boundary and final correctness contract. The broader lesson is simple: use metadata to understand the filtered search space, use vector retrieval where vector retrieval helps, and keep one authoritative answer at the end. That is what makes filtered vector search practical in a decoupled hybrid search engine.