Metadata-Filtered Retrieval for RAG

If your RAG demo has only ever run on 5 PDFs, you will probably never encounter metadata filtering. Once document volume grows - dozens of departments, thousands of product manuals, tickets spanning three years, and separate intranet and internet permissions - you will find that 80% of production retrieval requests are "conditional": search only within my department, only within the past month, only documents for this product, or only the content I am authorized to see.

None of these conditions can be handled by vector recall alone. Vectors can only tell you whether something is "semantically similar"; they cannot tell you whether a record belongs to Finance, has expired, or is visible to the user. This layer of precision requires metadata filtering. It is the sharpest dividing line between production RAG and demo RAG.

1. Why vector recall alone is not enough

naive RAG follows query embedding → 全库 top-k 余弦召回. It assumes that "the most relevant result is what should be answered," but in production, "relevant" is always preceded by a list of qualifiers:

• Time window: When searching for a "refund policy," the most similar result may be the version from three years ago - but it has already been retired. Vector recall returns it anyway, and the model confidently answers with a set of outdated rules.
• Product/business line: When searching for "how to export reports," customer service, finance, and BI products may all have documents about "exporting reports." Semantically, all of them look similar. The user only cares about the one they use.
• Department/tenant: If the HR knowledge base and legal knowledge base share an index, cross-department queries can immediately mix their contexts.
• Permissions: Internal documents visible to external partners must never enter the prompt. This is a compliance issue, not a minor question of whether an answer is accurate.

Vector recall alone has no way to express these four types of constraints. You can either ask the model in the prompt to "refer only to content within XXX time" - which it simply cannot do, because it uses whatever you provide - or push filtering down to the retrieval layer, so the model never sees non-compliant chunks. The latter is the production-grade approach.

In one sentence: embeddings determine "does it look similar?"; metadata determines "can it be used?". You need both legs to stand.

2. Attach metadata to chunks: plan from ingestion onward

Whether metadata filtering succeeds is decided the moment a chunk is ingested. What can be filtered during retrieval depends entirely on what was stored when it was labeled. Embeddings are the flesh; metadata is the skeleton - if the skeleton is not built properly, accurate flesh cannot stand on its own.

When each chunk is stored, it should at least carry these categories of metadata:

Two practical points:

First, chunk-level metadata = document-level inheritance + the chunk's own attributes. The department and product a document belongs to are inherited from its parent document; section titles and whether something is a "known issue/historical version" belong to the chunk itself. Merge these two layers in the ingestion pipeline rather than storing only document-level metadata.

Second, time fields must be stored as comparable types, not stuffed into strings. Store them as datetime in Qdrant payloads, and as timestamptz or a comparable string format in pgvector. I have seen "March 2024" stored verbatim; later range filtering became impossible, leaving re-ingestion as the only option.

3. Filtering capabilities in vector stores: pgvector / Qdrant / Milvus

All mainstream vector stores support filtering, but their expressive power differs greatly. Consider them in three tiers.

pgvector: direct SQL, the strongest expressive power. Because it is Postgres, filtering is simply WHERE; you can write arbitrarily complex SQL: ranges, combinations, JOINs with business tables, filtering after aggregation, and more.

-- 带元数据过滤的向量检索(pgvector)
SELECT id, content, metadata,
       embedding <=> $1::vector AS distance  -- 余弦距离
FROM chunks
WHERE department = 'finance'
  AND product   = 'billing'
  AND status    = 'published'
  AND updated_at >= '2024-01-01'
  AND visibility = ANY($2::text[])           -- 用户可见性集合
ORDER BY embedding <=> $1::vector
LIMIT 10;

This SQL means: among chunks for the Finance department's billing product that are published, from 2024 onward, and visible to the user, find the 10 most semantically relevant. Filtering and vector retrieval are written in the same statement and optimized together by the query planner. That is pgvector's killer feature.

Qdrant: payload filtering with a semantic API. Qdrant gives every point a payload (JSON), and filtering is expressed with Filter(must / must_not / should), close to Elasticsearch's DSL:

from qdrant_client import QdrantClient
from qdrant_client.models import Filter, FieldCondition, MatchValue, Range

client = QdrantClient(url="http://localhost:6333")

res = client.query_points(
    collection_name="docs",
    query=query_vector,
    query_filter=Filter(
        must=[
            FieldCondition(key="department", match=MatchValue(value="finance")),
            FieldCondition(key="product",    match=MatchValue(value="billing")),
            FieldCondition(key="status",     match=MatchValue(value="published")),
            FieldCondition(key="updated_at",
                           range=Range(gte="2024-01-01T00:00:00Z")),
        ],
        must_not=[
            FieldCondition(key="status", match=MatchValue(value="deprecated")),
        ],
    ),
    limit=10,
)

must is AND, should is OR, and must_not excludes matches. Together, they cover most scenarios. Combined with payload indexes (indexes on frequently filtered fields), Qdrant provides strong filtering performance, which is one of its strengths in mid-to-large-scale deployments.

Milvus: scalar fields + expressions. Milvus stores metadata in scalar fields and filters with SQL-like expr strings: expr='department == "finance" and product == "billing" and updated_at >= 1704067200'. Its expressive power falls between pgvector and Qdrant, while its strength is massive distributed deployments.

The selection rule in one sentence: choose pgvector for the strongest filtering expressiveness and a shared database with business data; choose Qdrant for mid-to-large-scale low latency plus a DSL experience; move to Milvus only for hundred-million-scale distributed deployments.

4. Filter first: narrow the set before vector retrieval

There are two filtering strategies, and production systems must distinguish them:

• post-filter: Perform vector top-k retrieval across the entire corpus first, then use metadata to remove non-matching results. The problem is severe: 9 of the top 10 results may fail the condition, leaving only 1 result, which is both inaccurate and insufficient. To compensate, you can only overfetch (retrieve top 100, then filter), increasing both latency and cost.
• pre-filter: Use metadata to narrow the candidate set to a subset that satisfies the conditions first, then perform vector retrieval within that subset. It is both accurate and fast, with no wasted overfetching.

In production, pre-filter is almost always the default. pgvector is naturally pre-filtered: when Postgres executes SQL, WHERE occurs before ORDER BY, so filtering takes place before vector sorting. The query planner automatically uses B-tree indexes to reduce the set before making vector comparisons. Qdrant is also designed around pre-filtering: payload indexes accelerate filtering before the vector index is queried. This execution order of "narrow first, then retrieve" is the fundamental reason filtering delivers performance benefits.

5. Performance benefits: smaller sets are faster and more accurate

"Will filtering make things slower?" Quite the opposite: filtering usually makes things faster.

The reason is straightforward: the computation required for vector retrieval is proportional to candidate-set size. Running ANN over 10 million records and running an exact scan over 50,000 records after filtering: the latter is often faster - especially when filter selectivity is high (removing 99% of the data). pgvector can skip the HNSW index and perform exact kNN over the filtered subset, making it both fast and accurate. Qdrant also has dedicated optimizations for highly selective filtering.

More importantly, filtering improves answer quality by removing noise. With a few fewer cross-department, expired, or irrelevant noise chunks in the top-k, the model has fewer opportunities to make things up. I have seen one case where, for the same question, adding time and product filters raised accuracy from 62% to 88% and reduced retrieval latency by 40% - because unfiltered ANN wandered through a large set, while filtering went directly to the target subset.

Filtering is not a burden on retrieval; it is a retrieval accelerator and denoiser.

6. Permission isolation: push security boundaries down to the retrieval layer

Permission isolation is the most serious application of metadata filtering. This is not optimization; it is a compliance red line.

The wrong approach is to retrieve from the full corpus and then "tell the model not to cite unauthorized content," or to trim results in the application layer after retrieval. The former is completely uncontrollable: the model has already seen confidential content, and prompt injection can extract it. The latter risks leaks: a bug in trimming logic can expose information.

The correct approach is to turn permission labels into hard retrieval filters and enforce isolation at the retrieval layer. Inject the user's visibility set (visibility / acl) as a must / WHERE condition so confidential chunks are not in the candidate set from the start and the model never sees them.

# 把用户权限作为强制过滤条件,硬注入
user_visible = resolve_acl(user)  # e.g. ["public", "internal", "g:finance-staff"]
query_filter=Filter(must=[
    FieldCondition(key="acl", match=MatchAny(any=user_visible)),
    # ...其余业务过滤
])

This way, even if the LLM is prompt-injected, it cannot "summon" content outside the user's permissions because the retriever never returned it. Security boundaries must be built at the lowest layer where they can be enforced, not on the courtesy of "the model promises not to look."

7. In practice: multi-condition combinations and index optimization

In implementation, index optimization determines real filtering-retrieval latency. A few key points:

On the pgvector side: Create B-tree indexes on frequently filtered fields (separate columns) or expression indexes for JSONB. For composite filters, pay attention to index column order: equality conditions first, ranges afterward, just as in traditional SQL tuning. Use HNSW with iterative_filter, which remains stable under highly selective filtering on large sets.

-- 高频过滤字段的复合索引(部门 + 产品 + 状态)
CREATE INDEX idx_chunks_filter ON chunks (department, product, status);
-- 时间字段单独索引,支持范围扫描
CREATE INDEX idx_chunks_updated ON chunks (updated_at DESC);

Move the most stable and most selective filter fields from JSONB into separate columns, where indexes and statistics are more accurate. This is the most valuable optimization once pgvector reaches the middle stage of use.

On the Qdrant side: Create payload indexes (keyword / integer / datetime) for frequently filtered payload fields. Put highly selective conditions in must first to help the engine narrow the set quickly. With multiple conditions, prioritize the one that eliminates the largest amount of data first (for example, tenant before product) for the highest pruning efficiency.

Avoid over-filtering: Too many combined conditions can reduce the candidate set to nearly 0, leaving top-k permanently unable to fill. Leave room for "broad admission, strict ranking" on key conditions: hard conditions (permissions and tenants) must be filtered, while soft conditions (time and relevance) can be weighted during reranking rather than being cut off at the retrieval layer.

Summary

• Metadata filtering is a production RAG necessity, not a nice-to-have. Eighty percent of production retrieval is conditional, and vector recall alone cannot express precise constraints such as time, product, and permissions.
• Filtering success is decided at ingestion time: every chunk needs source, department, product, time, permission, and status metadata; time fields must be stored in comparable types.
• The three vector stores each have their strengths: pgvector offers the strongest expressiveness through direct SQL, Qdrant provides robust payload + Filter performance, and Milvus excels at massive distributed deployments with scalar fields + expr.
• Filter first (pre-filter) is the production default: narrow the set before retrieval for both speed and accuracy, while removing noise along the way - filtering is an accelerator, not a burden.
• Permission isolation must be pushed down to the retrieval layer. Use hard filters to keep confidential chunks out of the candidate set from the start; do not rely on the model to behave.

Key takeaway: embeddings solve "does it look similar?" and metadata solves "can it be used?"; the former determines whether answers are accurate, while the latter determines whether the system can go live.