Multimodal Input Normalization Contract: Turning Untrusted Files into Reproducible Model Inputs

Passing file-extension and size checks only means the upload endpoint accepted the file; it does not mean the content seen by the model is deterministic. The same smartphone photo may contain EXIF rotation, Display P3 color, transparency, and enormous pixel dimensions; the first frame of an animation may be harmless while later frames contain the actual content; an audio container may declare 48 kHz even though its decoded timeline is discontinuous. If each worker directly invokes its own version of a library, caches, evaluations, and incident replays will not be operating on the same input.

Secure AI File Upload Pipeline for Next.js covers entry-point isolation, scanning, and permissions. This contract starts with quarantined objects and specifies which artifacts may enter the model. For basic guidance on selecting a Vision product approach, see Multimodal Vision Applications. Here, the question is not "what can the model see?" but "what exactly did the model receive?"

Store Raw and Canonical Objects Separately

Once accepted, a raw upload becomes read-only. Record its rawSha256, byte count, client-declared type, server-detected type, object version, tenant, and expiration time. Subsequent jobs do not overwrite the original object; instead, a pinned image generates canonical objects and manifests. Cache keys must include at least rawSha256 + normalizerVersion + policyVersion; using only a filename or URL allows same-name replacements and policy upgrades to retrieve stale results.

normalizerVersion: media-normalizer-3.4.1
runtimeImage: registry.example/media-normalizer@sha256:31c9...
input:
  rawSha256: 89b8...
  declaredType: image/jpeg
limits:
  maxBytes: 20971520
  maxPixels: 40000000
  maxFrames: 120
output:
  format: image/png
  colorSpace: srgb
  orientation: applied
  width: 1600
  height: 1200
  canonicalSha256: 2aa7...

When the declared type, extension, and magic bytes disagree, reject the file by default or route it for manual analysis. Never let the client-provided Content-Type select the decoder. Archives, SVG files, PDFs, and office documents must not masquerade as ordinary images; each complex format must use a separate sandbox with its own page-count, nesting, and external-link policies. The runtime image digest pins Pillow, ImageMagick, and system codecs, while the report also records the actual package versions. Artifact provenance, decoders, and policy versions should also be included in the bill of materials described in AI Supply Chain Security.

Decoding Is Itself Untrusted Computation

The decoder process must use read-only input, an empty temporary directory, no network access, a non-root user, and hard limits on CPU, memory, execution time, and output bytes. Read dimensions from the header first, then calculate the upper bound of width × height × channels × frames before allocating the full pixel buffer. An image that is only a few MB when compressed may still consume several GB after decoding. If a library emits a decompression-bomb warning, processing must fail; do not disable such warnings in bulk for compatibility.

Image normalization must run in a fixed order: fully decode, apply orientation, composite or reject transparency, convert to sRGB, remove unapproved metadata, resize with a deterministic algorithm, and then encode into a restricted set of formats. The order cannot be changed. For example, resizing before applying orientation changes the dimension check. Depending on the product contract, animated images must be rejected, retain a limited number of frames, or undergo explicit frame sampling; silently taking only frame zero is never acceptable.

The audio manifest records the actual sample rate, channels, sample format, duration, and timestamp continuity. Resample to a fixed format only when required by the target pipeline, and retain the algorithm version. For video, first probe the container and every stream, then limit duration, resolution, frame rate, track count, and subtitle attachments. This stage produces only a trusted timeline; actual frame extraction is performed by a separately versioned policy.

Prove Determinism and Rejection Boundaries with Golden Files

The test set must contain both normal samples and malicious edge cases: incorrect extensions, truncated JPEG files, enormous dimensions, small-file GIFs with many frames, unusual ICC profiles, EXIF rotation, transparent images, variable-frame-rate video, corrupted timestamps, silence, and multichannel audio. For every approved sample, store the canonical manifest and output hash; for every rejected sample, store a stable error category. After running the suite on x86 and ARM workers, any permitted differences must have been declared in advance.

A release gate can require all high-risk rejection cases to pass, golden canonical manifests to match 100%, repeated executions to produce identical output hashes, per-file CPU, memory, and output size to remain within limits, and errors never to echo original paths or media content. Comparing only whether the model's final answers are similar comes too late; the normalization layer must first prove that the input itself is explainable.

Logs record only digests, policies, dimensions, and error enums, never user media or publicly accessible object URLs. Canonical objects inherit the original object's tenant, permissions, and deletion deadline. Redaction, regional, and vendor-transmission boundaries must continue to follow Data Privacy and Compliance for AI Applications.

During Canary Rollouts, Run Both Versions but Keep One Source of Truth

When upgrading a decoding library or normalization policy, run the old version A and new version B on a de-identified golden set and a small number of approved production samples. Compare manifests, perceptual differences, downstream token usage, task quality, and resource consumption. Initially, keep A as the authoritative output while B writes only isolated diagnostics. Once differences have been classified, shift 5%, 25%, and then 100% of each media type to B.

If the rejection rate becomes abnormal, pixel differences exceed the threshold, downstream quality declines, or resource P95 exceeds its budget, routing must immediately revert to the old image and old policy digest. Stop reading from the cache namespace generated by the new policy, but retain it for diagnostics; old workers must not consume new canonical objects. A successful rollback must verify that new jobs once again receive A's canonicalSha256. In-flight jobs must either finish using their already bound version or be requeued.

The output of a normalization contract is not "a processed image." It is an evidence chain comprising raw facts, constrained transformations, deterministic output, inherited permissions, and executable version recovery. Only when this chain is stable can downstream cost optimization and quality evaluation compare the same input.

Further Reading

• Secure AI File Upload Pipeline for Next.js
• Multimodal Vision Applications
• AI Supply Chain Security
• Data Privacy and Compliance for AI Applications