Your RAG Metadata Filter Isn't Filtering: Silently Serving the Wrong Tenant's Data

The scariest bug in a retrieval system is not the one that crashes. It is the one that works. You add a metadata filter so that a query for tenant A only ever retrieves tenant A's documents. The code runs, no exception, results come back, the answers look right. You ship it. Months later someone audits the logs, or a customer says "why does your assistant know about a company that is not mine", and you discover the filter was never applied. Every query has been retrieving from the whole corpus the entire time.

This is a real, reported class of bug. One concrete instance:

[Bug]: llama index OData filter for Azure AI Search vector store is not filtering the chunks

The filter object is constructed, passed to the retriever, the retriever calls the vector store, the store returns chunks. Nothing fails. But the filter predicate never made it into the actual query the store executed, so the chunks come back unfiltered. The framework quietly dropped your isolation boundary and told you nothing.

The thesis: a filter that returns results is not a filter that filtered. In a multi-tenant RAG system, a silently ignored metadata filter is a data-isolation breach that passes every casual test, and the only thing that catches it is an assertion that fails when a foreign document appears.

Why the filter gets dropped without an error

Frameworks like LlamaIndex and LangChain give you a store-agnostic way to express a filter: something like "metadata field tenant_id equals acme". That is convenient because you write it once and it is supposed to work across Pinecone, Chroma, Weaviate, Azure AI Search, pgvector and the rest. But each of those stores has its own filter language. Azure AI Search wants an OData filter string. Pinecone wants a JSON filter dict. pgvector wants SQL. Chroma wants a where clause.

So between your generic filter and the store there is a translation layer that converts one into the other. That layer is where isolation goes to die, for three reasons:

• Unsupported operator or field. The store or its client version does not support the exact predicate you expressed, and instead of raising, the adapter emits a query with the filter omitted.
• Wrong field path. The metadata was indexed under one key but the filter references another (a nested field, a prefixed field), so the predicate matches nothing meaningful and is effectively skipped.
• Version drift. The store's filter API changed, the adapter was written for the old one, and the new endpoint ignores an unrecognised parameter rather than rejecting it.

In all three, the outbound call is well-formed enough to succeed. Vector search is designed to always return the nearest neighbours it can find; a filter is an optional narrowing, so a dropped filter degrades gracefully to "return everything", which is precisely the wrong default for security.

The test that catches it: a negative assertion on isolation

You cannot spot this by reading a few answers, because the wrong answers look plausible. You need a test whose only job is to fail the moment a foreign tenant's document appears in the results.

Step 1: index two tenants with a document unique to each

from llama_index.core import VectorStoreIndex, Document

docs = [
    Document(text="Acme Q3 revenue was strong across all regions.",
             metadata={"tenant_id": "acme"}),
    Document(text="Acme is launching a new logistics product line.",
             metadata={"tenant_id": "acme"}),
    # A canary that must NEVER appear in an acme-filtered result:
    Document(text="Globex is negotiating a confidential acquisition.",
             metadata={"tenant_id": "globex"}),
]
index = VectorStoreIndex.from_documents(docs, vector_store=store)

Step 2: query with the filter and assert nothing foreign comes back

from llama_index.core.vector_stores import MetadataFilters, MetadataFilter, FilterOperator

def test_tenant_filter_excludes_other_tenants():
    filters = MetadataFilters(filters=[
        MetadataFilter(key="tenant_id", value="acme", operator=FilterOperator.EQ)
    ])
    retriever = index.as_retriever(similarity_top_k=10, filters=filters)

    # Deliberately query with words from the GLOBEX document, the hardest case:
    nodes = retriever.retrieve("confidential acquisition negotiation")

    returned_tenants = {n.metadata.get("tenant_id") for n in nodes}
    assert returned_tenants == {"acme"}, f"leaked tenants: {returned_tenants}"
    assert all("Globex" not in n.get_content() for n in nodes)

The query text is chosen to be most similar to the forbidden document. If the filter is silently ignored, the Globex canary is the top hit and the test fails loudly. If the filter works, that document is excluded despite being the closest match, which is exactly the behaviour you want to prove.

Step 3: prove the corpus is not just empty

A filtered query returning zero foreign documents could mean the filter works, or it could mean the index is empty and nothing was ever going to come back. Rule out the second with a control:

def test_globex_document_is_actually_retrievable_unfiltered():
    retriever = index.as_retriever(similarity_top_k=10)  # no filter
    nodes = retriever.retrieve("confidential acquisition negotiation")
    assert any(n.metadata.get("tenant_id") == "globex" for n in nodes), \
        "canary not in corpus; the filtered test proves nothing"

Now the pair is airtight. Without the filter, the Globex document is retrievable. With the filter, it is gone. That is real exclusion, not an accident of an empty index. This is the same discipline I argue for in evaluating LLM features before they touch a customer: assert on the property that matters, do not eyeball the output.

Step 4: test the native store query directly to localise the bug

If the framework test fails, you need to know whether the store cannot filter or the framework failed to translate. Fire the store's own native filter query, bypassing the framework entirely.

# Example: Azure AI Search native OData filter, no framework in the path
results = search_client.search(
    search_text=None,
    vector_queries=[VectorizedQuery(vector=embed("confidential acquisition"),
                                    k_nearest_neighbors=10, fields="embedding")],
    filter="tenant_id eq 'acme'",   # the native OData predicate
)
tenants = {r["tenant_id"] for r in results}
assert tenants == {"acme"}

If the native query filters correctly but the framework path does not, the bug is confirmed to be in the translation layer, and your options are: upgrade the adapter, express the filter differently so it translates, or drop to the native client for the retrieval call. If even the native query leaks, the metadata was not indexed as a filterable field, which is a schema problem at index time, not a query problem.

What people get wrong

Trusting "it returned results" as proof the filter worked. Returning results is the failure mode, not evidence against it. A dropped filter returns more results, not fewer. The presence of output tells you nothing about whether the boundary held.

Testing only the happy tenant. Querying as tenant A and seeing tenant A documents proves nothing, because tenant A documents would come back with or without the filter. The test has to assert on the absence of tenant B, with a query engineered to favour tenant B.

Filtering in application code after retrieval. Some teams "fix" this by retrieving unfiltered and then discarding foreign documents in Python. That leaks anyway: the foreign documents were loaded into your process, may appear in logs or traces, and if you forget the post-filter on one code path the data is right there. Filter at the store, and test that the store did it.

Assuming the framework abstraction is the source of truth. The abstraction is a convenience, not a guarantee. Its correctness depends on an adapter for your specific store and version. When isolation is on the line, verify against the store, as covered in the retrieval architecture in building RAG over your ERP data.

When it is still broken

• The native query filters but production still leaks. Your production retriever is not using the filter you think it is. Trace the actual filter object at the call site; a default retriever built elsewhere may not carry it.
• Filter works for string equality but not for lists or ranges. Many adapters translate EQ but silently drop IN, GTE or nested-field predicates. Test each operator you rely on separately; do not assume one working operator means all work.
• Metadata is there but not filterable. Some stores require a field to be explicitly marked filterable at index-creation time. If it was indexed as content-only, filters on it are ignored. Recreate the index with the field declared filterable.
• It regresses after a dependency bump. Translation-layer behaviour changes between versions. Pin your vector-store client and framework versions, and keep the isolation test in CI so a future upgrade that reopens the hole fails the build instead of shipping.

The uncomfortable takeaway: in a multi-tenant retrieval system, "no error and plausible answers" is the exact signature of a silent data leak. Treat every metadata filter that guards a tenant boundary as untrusted until a negative assertion has proven it excludes the data it is supposed to exclude.