RAG Index Artifact Releases: Signed Manifests, Reproducible Builds, and Atomic Rollbacks

knowledge-prod-v7 cannot identify the source snapshot, parser, chunk parameters, embedding weights, or ACL coverage; shards may be missing from an upload, or an alias may be manually changed while the release still appears successful. RAG Incremental Index Consistency constrains single-document events; this article turns an entire batch output into a verifiable immutable release.

A Manifest Is a Release API, Not a Note

Define rag.index.release.v1; its serialization rules and field semantics are frozen with the version. A minimal manifest should include:

{
  "schema": "rag.index.release.v1",
  "releaseId": "idx_20260718_01",
  "sourceSnapshot": "sha256:...",
  "parserImage": "sha256:...",
  "chunkerConfig": "sha256:...",
  "embedding": {
    "model": "vendor/model@sha256:...",
    "dimensions": 1024,
    "metric": "cosine"
  },
  "aclSchema": 3,
  "documents": 482091,
  "chunks": 2948812,
  "shards": [{
    "mediaType": "application/vnd.codenodex.rag-shard.v1",
    "digest": "sha256:...",
    "size": 817263,
    "uri": "store://cas/sha256/...",
    "objectVersion": "gen-8841"
  }],
  "coverageReport": "sha256:...",
  "evaluationReport": "sha256:..."
}

Every shard uses an OCI descriptor-style mediaType + digest + size and binds an immutable object-storage version/generation; the URI is only a location. Staging may only conditionally create content-addressed objects and must not overwrite them; mutable latest never enters the manifest. The source snapshot includes connector watermarks, resource versions, and tombstone digests; the parser, OCR, chunk configuration, metadata schema, embedding dimensions, and metric are all part of the index ABI, and any change creates a new release.

Sign the Build Identity; Runtime Identity Only Verifies

After the builder uploads every immutable shard, it calculates SHA-256 over the canonical manifest bytes and signs it. Sigstore verification pins the issuer, subject, certificate identity, and Rekor policy; offline KMS pins the key ID, algorithm, and rotation window, rejecting a manifest that specifies a remote public key. SLSA provenance records the builder and inputs, while the RAG manifest separately binds counts, ACL coverage, and evaluation.

The promotion service has permission only to read artifacts, verify signatures, and switch aliases; it cannot build or modify shards. The builder can write staging but cannot switch production. Human approval binds to the manifest digest, not a mutable release name. This causes replacement of a same-named file after approval to fail before signature verification, so the artifact identity boundary in AI Supply Chain Security truly covers index data.

Reproducible Does Not Mean Floating-Point Results Are Byte-Identical

Inputs, parsing, and chunking should replay deterministically, producing the same IDs, text hashes, metadata, and order. When embeddings differ numerically across hardware, either fix the environment and publish the actual shard digest, or define top-k overlap rate, ranking correlation, and maximum distance deviation; do not pretend they are byte-for-byte reproducible.

The acceptance verifier does not trust the builder's success: it opens the exact object version from each descriptor, checks the actual size and media type, and streams a recomputation of the digest. It then confirms that the release ID, dimensions, metric, and aclSchema agree, and compares the source, index, and deletion sets. Permission coverage must be 100%; unknown schemas, orphans, duplicate IDs, and incomplete shards must all be zero. The source version and chunk hash required by RAG Citation Provenance also come from this artifact.

Promotion Is a Conditional Update, Not a Directory Copy

Production resolves only an active-release containing a manifest digest and a monotonically increasing generation. Promotion performs CAS only when the approval generation is unchanged and the candidate passes signature verification and gates. After verification, workers pin a local read-only file handle/snapshot; if they cannot pin locally, every open must specify the object version in the signature, then recheck the size and digest before passing it to the index reader. Never verify a mutable URI, close it, and then open a same-named path; a single request must not mix reads across generations.

Before switching, run fixed-fact queries, empty-result queries, exact-string queries, cross-tenant negative samples, deleted documents, and citation expansion, then compare the recall, latency, and quality thresholds in RAG Embedding Migration. Canary releases use stable tenant buckets, and reports bind to the candidate digest. If the manifest is the same, repeated promotion is an idempotent no-op; if the name is the same but the digest differs, reject it directly.

Fault Injection and Atomic Rollback

Exercise shard tampering, forged size/media type, replacement of a same-named object after signature verification, an incorrect generation, missing shards, expired certificates, missing coverage, and a crash after CAS. A candidate may only remain STAGED or switch completely; a new worker that cannot revalidate a descriptor must refuse readiness, while an already-pinned and verified read-only handle may serve during the specified period but cannot skip digest checks.

Retain at least the previous verified release until the rollback window ends. When quality or latency degrades, use CAS to point back to the old manifest digest and increment the generation without modifying either the new or old shards; current facts such as deletion or permission revocation are still enforced by the live deny waterline, and a rollback index must not resurrect denied resources. After rollback, replay the triggering queries and citation audits to confirm every worker converges on the same generation. The workflow can draw from Model Serving Canary Releases, but index rollback additionally requires that source versions and permission state do not regress.

Further Reading

• RAG Incremental Index Consistency
• RAG Embedding Migration
• RAG Citation Provenance
• AI Supply Chain Security