Multi-LoRA Serving Isolation: Adapter Identity, VRAM Quotas, and Cross-Tenant Mix-Up Prevention

Multi-LoRA allows a single base-model process to select different low-rank adapters for each request, avoiding a complete copy of the weights for every workload. Saving VRAM does not provide isolation by itself: a client that forges an adapter name may access another tenant’s capabilities, hot-loading an unreviewed file may apply incorrect tensors, and omitting a version key from batching or caching may cause tenant A to receive tenant B’s behavior. Sharing may occur only in explicitly shareable base weights and compute layers; identity, state, and results must remain separate for every request.

Adapters Must Enter a Trusted Registry First

Production requests cannot contain arbitrary disk paths or download URLs. After the training pipeline produces an adapter, it writes the adapter weights, PEFT configuration, base-model digest, target modules, rank, alpha, tokenizer dependencies, training-data version, evaluation report, and signature to a read-only registry. The platform scans the format and size, verifies that every tensor name, shape, and target layer belongs to the declared base model, and then assigns an internal immutable digest.

adapterId: support-zh
adapterDigest: sha256:aa71...
baseModelDigest: sha256:90e4...
peftType: LORA
rank: 16
targetModules: [q_proj, v_proj]
ownerTenant: t-1042
maxReplicas: 4
state: APPROVED

A business alias alone is insufficient: support-zh may point to a new version, but an active request must remain bound to the digest resolved at that time. Startup must fail when an adapter does not match the base model; the runtime cannot attempt a “best-effort” load. LoRA/QLoRA Fine-Tuning explains the training principles, while an online platform must additionally establish artifact provenance, approval, and runtime boundaries. The signatures and bills of materials described in AI Supply Chain Security apply equally to adapters.

Request Identity Determines the Adapter, Not Client Parameters

The gateway resolves the permitted adapter from the verified tenant identity, product entitlement, and server-side policy, then generates a signed internal routing context. The worker does not trust adapter_id from a public header. The authorization table is deny-by-default: a missing mapping returns an explicit error instead of falling back to another tenant’s or the “most recently used” adapter. Administrator debugging also uses short-lived, auditable impersonation authorization.

At runtime, every request records tenantId + baseDigest + adapterDigest + tokenizerDigest, and the same identity propagates through queueing, batching, cancellation, and metering. Dynamic batching may send requests for different adapters into the same scheduling pass, but index mappings, output buffers, errors, and cancellation signals must remain correctly associated with each request. If the runtime does not reliably support mixed batching, group requests by adapter instead of experimenting for additional throughput.

Isolation tests must actively submit invalid combinations: tenant A guessing tenant B’s alias, a revoked version, the wrong base model, concurrent adapter switching, a request canceled mid-execution, and a worker restart. The expected result is not “correct most of the time”; every unauthorized combination must be rejected before loading, and neither observability nor responses may expose another tenant’s names or paths. Authorization propagation can reuse the trusted-context principles from MCP Multi-Tenant Isolation.

Account for Both VRAM and Caches by Digest

Set per-adapter limits for weight size, rank, concurrent sequences, resident adapter count per tenant, and the total number loaded globally. Loading proceeds through a bounded queue and reserves VRAM. At the high-water mark, first evict the least-recently-used versions that have no active references, and never unload weights still held by requests. A high-traffic tenant cannot crowd out every other adapter by continually creating new versions; enforce resident shares and loading rates per tenant.

Observe the base model, adapter weights, runtime workspace, and KV Cache separately. At minimum, report each digest’s loading duration, loading failures, resident bytes, active requests, hits/swap-ins, eviction reasons, and per-token latency, while external tenants can see only their own aggregates. High-cardinality digests must not become uncontrolled labels in global metrics; record them in traces or restricted logs instead.

KV Cache, prefix cache, and result cache keys must include the adapter digest. LoRA changes layer outputs, so KV produced by another adapter cannot be reused even when the input tokens and base model are identical. If the tokenizer or chat template varies by adapter, include it in the key as well. Platform-level shares and atomic accounting can be implemented together with AI Platform Tenant Quota Isolation.

Evaluate and Release Every Adapter Independently

When upgrading a shared base model, run the compatibility matrix for every approved adapter rather than validating only the most popular one. Gates must include each adapter’s golden tasks, a control using the base model without an adapter, mismatch rejection, concurrent mixed batching, long contexts, cold loading, and memory pressure. Example thresholds can include zero authorization errors, a structured-output success rate no lower than the previous version, P95 cold loading below 8 seconds, resident VRAM measurement error below 5%, and zero response-attribution errors under 32 mixed concurrent requests.

During release, canary the alias from adapterDigest=A1 to A2 using stable session routing at 1%, 10%, and 50%, while retaining A1 weights and capacity. If the task success rate falls by more than 1 percentage point, loading failures exceed 0.5% for three consecutive windows, or any identity mismatch occurs, stop new traffic and restore alias resolution to A1. Depending on risk, drain sessions already bound to A2 or replay them from their original messages. Traffic progression can follow Canary Rollouts for Model Serving, but the rollback unit must be a validated combination of the base model and adapter.

When revoking an adapter, first disable new resolution, then wait for its references to reach zero, clear related caches, and unload it from the nodes. The registry retains the audit record but no longer serves the artifact for download. Training-data issues or tenant termination must both allow affected requests to be identified by digest. Multi-LoRA reliability is not measured by “how many adapters fit on one GPU,” but by whether every selection still has a trusted identity, deterministic artifact, independent resource ledger, and verifiable rollback after base compute is shared.

Further Reading

• LoRA/QLoRA Fine-Tuning
• AI Supply Chain Security
• MCP Multi-Tenant Isolation
• AI Platform Tenant Quota Isolation
• Canary Rollouts for Model Serving