RAG Data Source Admission: Quarantined Parsing, Trust Tiers, and Revocable Promotion

Enterprise RAG connectors are often configured with an "ingest anything they can read" mindset: shared cloud-drive folders, ticket attachments, web pages, and emails are automatically parsed, embedded, and added to production retrieval as soon as they appear. An attacker need only upload a PDF containing hidden instructions or take over a long-unmaintained web page for malicious content to enter the model context. Prompt Injection Defense covers runtime defense in depth; this article addresses an earlier governance question: which source, which version, for which purpose is approved to enter which index.

A Source Registry Before the Connector

Each source must first be registered under a source.v1 contract: a stable source ID, owner, tenant, connector identity, allowed paths, data classification, legitimate purposes, allowed MIME types, crawl frequency, maximum object and expanded sizes, retention period, trust tier, and emergency contact. Credentials may read only the declared paths; a connector's technical ability to traverse an entire site does not make every space authorized corpus.

Source versions must also bind a normalized locator, upstream revision/ETag, retrieval time, raw-byte SHA-256, and connector image digest. W3C PROV-style lineage connects the raw object, parsing activity, and chunks, allowing subsequent RAG Citation Provenance to answer: "Who produced this content, from where, and with which parser?" The same URL with a changed digest is a new revision and must not silently overwrite an approved version.

A trust tier describes processing requirements, not that "the content is certainly true." For example, PUBLIC_UNVERIFIED may support only low-risk search, INTERNAL_MANAGED requires a verified owner and ACL, and REGULATED additionally requires human approval and a shorter cache. User-uploaded content does not automatically become a trusted policy source merely because it resides on a corporate domain.

Raw Objects First Enter a Non-Executable Quarantine

A connector may write only to an append-only quarantine bucket and ingestion ledger; it has no production-index permissions. Quarantine objects are digest-addressed, and the server ignores client-supplied filenames and MIME declarations; it first checks magic bytes, size, compression expansion ratio, file count, nesting depth, encryption status, and allowed types. Macros, scripts, embedded objects, external-link fetching, and parser network access are disabled by default.

Parsers run in sandboxes with no credentials, no outbound network access, read-only input, temporary output, and CPU, memory, time, and byte quotas. The pipeline can follow the isolation and scanning principles in Next.js AI File Upload Security, but they apply equally to backend connectors and must not be skipped simply because the upload did not come from a browser. Archive bombs, malformed PDFs, or OCR limit overruns must produce explicit REJECTED_* results rather than automatically falling back to "read as much as possible as plain text."

Parsing output retains pages, offsets, and structure, and does not execute links or tool calls in HTML/Markdown. Detect hidden text, extreme fonts, zero-width characters, prompt-injection patterns, credential-like patterns, and large encoded payloads; these detections route content to human review and cannot promise to identify all malicious semantics. NIST AI 600-1's requirements for measuring, recording, and handling generative AI risks can serve as a risk-register baseline.

Promotion Requires Both Purpose and Content Gates

The state machine is fixed as DISCOVERED -> QUARANTINED -> PARSED -> REVIEWED -> PROMOTED, with side states REJECTED and REVOKED. Every transition records the actor, policy revision, input/output digest, and reason code. High-trust automatic sources may be approved by a policy service; external web pages, user uploads, and sources that affect policy answers require owner review. Approval binds to the exact revision digest and cannot permanently trust future content after approving a URL.

Promotion creates an immutable source release: allowed resource revisions, trust tier, ACL policy, purpose set, parser and chunker digests, risk report, and expiration date. The production index builder accepts only signed releases and does not scan the quarantine directory. Content with purpose=support cannot be reused by a marketing assistant merely because it has the same tenant; RAG Retrieval Access Control must filter purpose and trust tier in addition to subject ACLs.

All retrieved natural language retains the UNTRUSTED_RETRIEVAL taint regardless of trust tier. The tier determines only ingestion, search, and review requirements; human approval, citation labels, delimiters, or a "trusted source" cannot elevate content into control instructions. Tainted output must not determine tool names, calling identities, permission scopes, resource targets, policies, or approval outcomes.

When the model needs to propose a business action, the executor accepts only business data validated by a deterministic decoder, versioned JSON Schema, and field allowlist; the subject, tenant, tool, and target resource are resolved from trusted server state and re-authorized or approved. Unknown fields and model-supplied control fields are rejected directly, and free text must not enter commands, URLs, or selectors without sink validation. Source admission cannot replace the execution boundary described in Agent Untrusted Data Taint Propagation.

Verify with Recomputable Gates

For a source release, recompute the registered resource set, quarantine digest set, successfully parsed set, approved revision set, and indexed resource set. Gates require indexed - approved = 0, zero unknown owners, zero expired approvals, 100% ACL/purpose/trust field coverage, and every chunk to trace back to its raw digest. Sampled citations must open the same revision, not the currently changed web page.

The red-team corpus includes indirect prompt injection, documents requesting changes to tools/tenants/targets/roles, white text, malicious external links, same-URL replacement, connector overreach, and revoked owners. Assert that rejected objects do not enter candidates or cache, and that malicious natural language which was promoted cannot alter server-side tools or authorization parameters; when the scanning service is unavailable, processing stops in quarantine. Reducing false positives may only change policy and replay the pipeline; it must not fail open.

Revoke Sources Rather Than Delete Audits

When an owner leaves, a license changes, an upstream system is compromised, or content is misleading, submit source.revoked.v1: first add the source/revision to a real-time deny set, terminate in-flight contexts, invalidate query, answer, and citation caches, then remove the related chunks from the next index release. Events use monotonic revisions, so replay by an old connector cannot promote them again. RAG Incremental Indexing Consistency handles physical convergence, while the real-time deny set keeps content invisible during the revocation window.

An erroneous revocation can reapprove the same digest and generate a new promotion revision; do not simply delete the revoke record or mark the old approval valid again. If a new scanner produces false positives, roll back to the previous parser/policy version, reparse from immutable raw objects, compare output digests, and pass through the gates again. Retain minimal audit metadata and digests, but do not retain original text beyond the policy retention period.

Further Reading

• Prompt Injection Defense
• Next.js AI File Upload Security
• RAG Retrieval Access Control
• Agent Untrusted Data Taint Propagation