RAG Permission Metadata Migration: Dual Writes, Coverage Proof, and Default-Deny Cutover

Early knowledge bases often write roles: ["employee"] directly into each chunk. Later, when project teams, external collaborators, and document-level exceptions are needed, it is tempting to change this to tenantId + policyId + classification. If the reader is released first, old chunks with missing fields may be treated as public; if the writer stops writing the old fields first, a rolled-back reader can no longer see new documents. This is not a vector-quality cutover like RAG Embedding Migration, but a schema evolution in which an error at any step can broaden access.

Freeze Two Complete Contracts First

Define immutable versions for the old and new metadata; do not infer them from the presence of fields:

{
  "aclSchema": 3,
  "tenantId": "t_42",
  "resourceId": "doc_9",
  "policyId": "pol_71",
  "policyRevision": 12,
  "classification": "internal",
  "validFrom": "2026-07-18T00:00:00Z",
  "sourceVersion": 884
}

aclSchema, tenantId, resourceId, and policyId are all required for v3; unknown versions, null values, type errors, and tenant mismatches must never be retrievable. The v2 -> v3 mapping function is pinned to a fixed code digest and defines an unconvertible state, QUARANTINED; it must not use classification=public as a default. Authorization semantics still follow RAG Retrieval Access Control; the migration only changes its verifiable representation.

Expand, Backfill, Cut Over, Contract

First create v3 fields or a new index, and confirm that the vector store supports the target field types, arrays, and filter operations. As with PostgreSQL ALTER TABLE, add a compatible representation and remove the legacy one in separate releases; the vector service's support for equality, set, and compound filtering on metadata must also be probed with real queries, not inferred from SDK types.

Second, do not remotely dual-write inside a source database transaction. A transaction assigns a monotonically increasing materializationGeneration to the resource and commits a complete desired snapshot, containing sourceVersion, policyRevision, and a payload hash, together with its corresponding outbox record. The v2 and v3 projectors conditionally write only when the incoming generation exceeds the remote generation; they must not compare the two version axes independently. The coordinator reads both projections and, after verifying that the resource, generation, both versions, ACL, and content digest match, writes a per-resource committed marker bound to that same set of values; a timeout or lost response on either branch must not make that generation current. Deletion tombstones also take effect only at a higher generation, and their rules share the same monotonic comparison as RAG Incremental Indexing Consistency.

Third, the reader first executes v2 and v3 filtering in shadow mode. For each request, record a digest of the candidate resource ID set, permitted differences, rejection reasons, and schema coverage, but do not write document bodies into the difference log. Only expected policy changes may produce set differences; any unapproved resource in v3_allow - v2_allow is a security blocker. The formal cutover pins routing by tenant: within a request, an empty v3 result must never automatically fall through to an unfiltered v2 query.

Finally, stop v2 writes and remove old fields only after the full rollback window. Old readers, old writers, and old backfill jobs should all be rejected by version admission control, preventing a replay job six months later from recreating v2-only chunks.

The Coverage Ledger Must Be Recomputable Item by Item

Equal document totals are not proof. A coverage run first freezes the source CDC watermark S, ACL CDC watermark A, and candidate index generation G, then records, by source, tenant, classification, and policy, the resource count, chunk count, missing v3 entries, quarantined entries, orphaned entries, and a digest of sorted (resourceId, sourceVersion, policyId, chunkHash). The report must be bound to S/A/G and generate diagnosable difference sets from the source, authorization store, and index at the same watermarks; it must not subtract three current values from different points in time.

Before alias CAS, reread projector offsets, candidate generation, and uncommitted markers: cut over only if v2 and v3 each have lastSourceApplied >= S and lastAclApplied >= A, G is unchanged, and all desired revisions inside the fence are committed. Changes after the fence still proceed through continuous dual projection and can be selected as current by the reader only when their own marker has completed; if a watermark or generation changes, discard the report and rerun it.

Run negative retrieval against a fixed permission matrix: no tenant, forged tenant, removed member, cross-project user, expired external account, unknown schema, missing policy, and an old policy revision. Each query must assert not only that the final answer contains no secrets, but also that vector candidates, rerank inputs, traces, and citations contain no unauthorized resource ID. The cache-key regressions in MCP Multi-Tenant Isolation must also be rerun, because schema version and policy revision have become new cache-affecting factors.

Failure Injection Matters More Than a Smooth Backfill

In preproduction, inject a successful v3 conditional write with a lost response, projector crashes, cross-axis out-of-order pairs (source=2, policy=1) and (source=1, policy=2), late deletions, null fields, legacy mapping shards, stalled offsets, and alias CAS conflicts. The hard invariant is this: no projection without a committed marker for the matching generation, or with aclSchema != 3, may enter the v3 production context; degraded availability may pause processing, but filtering must not be disabled.

Release gates can be explicit: 100% coverage for current resources, 0 unknown schemas, 0 orphan chunks, and 0 leaks across all negative samples; every allow-set difference has an approval ticket; P95 filtering latency and empty-result rate are within budget. Sampled recall and answer quality are only additional gates; they cannot replace set proof.

Roll Back the Reader, Not Authorization Facts

Before the cutover, retain the v2 reader, v2 metadata, and the previous index alias. If v3 falsely denies access or latency gets out of control, pin affected tenants' routing back to v2 while preserving dual writes and freezing schema contraction; the rollback action itself records the operator, manifest digest, and tenant set. If v3 unexpectedly allows access, first disable production queries for that version and use default-deny, then fix the mapping; do not return to a known unsafe path to restore answer rate.

Rollback cannot undo permission changes that have already occurred, nor resurrect deleted documents from the old index. After recovery, rerun the coverage ledger, negative matrix, and Model Serving Canary Rollout-style staged traffic shifting. The true sign that a permission schema migration is complete is not that the v3 index has been built, but that every current chunk can prove that it belongs to a known, current authorization contract that defaults to deny.

Further Reading

• RAG Retrieval Access Control
• RAG Embedding Migration
• RAG Incremental Indexing Consistency
• Model Serving Canary Rollout