Per-Commit Parsing Budget for Streaming AI Markdown: Block Sealing and Semantic Gates

Streaming Markdown can easily hide quadratic work: the first chunk parses 100 characters, the second parses 200, and so on until the entire response is repeatedly recomputed. Front-End LLM Streaming Implementation and RAG Streaming UI have already covered buffering, stable segments, and deferred enrichment, while Streaming UI Commit-Rate Budgets limits the number of commits per second. This article starts after the complete record has been validated and a batch of text has been selected for a commit; it constrains only the scanning, parsing, and enrichment cost of that commit and the final CommonMark semantics.

Prove Semantics with Identical Content and Different Chunking

The benchmark body is fixed at 100 KiB and contains 30 headings, 20 code fences totaling 600 lines, 3 tables, 80 reference links, Chinese combining characters, and one deliberately late-closing code fence. Split the same body into single-character chunks, random chunks of 1 to 64 bytes, line-based chunks, and 4 KiB chunks, then replay it at both 30 chunks per second and in bursts of 200 chunks. The report records the body SHA-256, chunking seed, parser and plugin versions, React 19.2.4, browser build, and CPU slowdown multiplier.

The gates require the final normalized HTML and accessible text summary to be identical under every chunking scheme; no characters may be lost or duplicated; no individual main-thread task may exceed 50 milliseconds; parsing-slice P95 may not exceed 8 milliseconds; the Profiler rendering subtree's actualDuration P95 may not exceed 12 milliseconds; P95 latency from receiving a character to displaying it safely may not exceed 150 milliseconds; and the cumulative number of parsed characters may not exceed 2.5 times the final body length. The final requirement directly prevents parsing from scratch every time, even if a fast machine has not yet produced any long tasks. TypeScript Async Iteration can organize the input, but asynchronous syntax alone does not yield enough budget to the event loop.

The Tail Is Mutable; Sealed Blocks Are Immutable

The rendering model is divided into sealedBlocks and openTail. The parser seals a block only after establishing a local block boundary, such as a paragraph followed by a blank line, a complete heading, a closed code fence, or a semantic boundary provided by the protocol; unclosed fences and lists remain in the tail. CommonMark link reference definitions can appear after their references, so “never reparse after sealing” applies only to a streaming subset that explicitly disables this kind of global syntax. If the product supports reference links, the parser records dependencies from labels to blocks and invalidates only the affected blocks when a definition arrives; at completion, it always performs one canonical full parse and uses that as the final tree. The tail is initially displayed as escaped plain text or restricted Markdown, then replaced with complete nodes once the syntax is known.

You cannot simply split strings on newlines: fences, lists, tables, and inline markup can span chunks. The boundary detector must be a stateful incremental scanner and must cap the maximum unsealed length eligible for rich-text parsing, such as 16 KiB; after the limit is exceeded, the content remains plain text, but the scan cursor continues advancing with a constant amount of state. If a closing fence does not appear until after 64 KiB, the canonical parse at completion must still recover the code-block semantics. It cannot permanently treat the content as ordinary text or repeatedly scan the entire document to find the closing fence. Network framing and structural decoding continue to follow Parsing Streaming Structured Data in the Browser; sealing here determines only the view syntax and does not guarantee transport integrity.

Only after a stable block's initial render should enrichment be queued. Syntax highlighting, mathematical formulas, copy buttons, and link previews each have an independent task ID, and their results are bound to the block digest. If a block is replaced or the run is canceled, stale results are discarded. Each enrichment task processes one bounded block and yields to the event loop between tasks. If highlighting a single block may still exceed 50 milliseconds, send it through the existing Web Worker Concurrency path, and let the main thread apply only results with matching digests. The Worker does not change the budget; it only moves parallelizable CPU work off the interaction thread.

useDeferredValue can allow the enriched tree to lag behind the latest plain text, but it does not cancel network requests or automatically move synchronous parsing into the background. The interface must explicitly retain the latest safe text, while the deferred version only enhances its presentation; a stale deferred tree must not revert copied content or reference targets to an older version. Code-highlighting styles must reserve layout dimensions to avoid layout shifts when nodes are replaced.

Instrument Every Stage, Not Just FPS

Record received for each chunk and scan_done, safe_commit, enrich_start, and enrich_done for each sealed block, then aggregate them with performance.measure. PerformanceObserver simultaneously collects longtask entries; a React profiling build records the target subtree's actualDuration; and a browser trace attributes DOM, layout, and paint work. During the stream, tests continuously type, select text, scroll, and expand references, while injecting a 20-millisecond delay at three points—parsing, committing, and highlighting—to verify that the queue remains bounded and urgent input can still be painted.

Create two additional 64 KiB code-fence fixtures: one never closes, while the other closes only after more than 16 KiB. During streaming, both maintain a bounded plain-text tail, and the scan cursor advances without backtracking. At completion, the first preserves the original text and marks its formatting as incomplete; after the final canonical parse, the second must produce the same code block as a one-shot CommonMark parse. Next, seal a reference before its reference definition arrives and assert that the dependent block is invalidated and the final link semantics match. Do not append symbols to manufacture a seemingly valid tree.

Roll out with dual computation and single rendering: 1% of traffic continues to display the old renderer, while sampled sessions generate a digest of the new canonical tree in the background and compare the final result and cost without mounting both DOM trees on the page. Once the digests match, display the new path to 1% of traffic and observe long tasks, visible latency, memory, and copied results. If any body digest differs, the proportion of tasks exceeding 50 milliseconds rises, the open tail exceeds its limit, or P75 INP increases by 20%, switch back to the conservative “plain text during streaming, parse once after completion” path. Rolling back neither reconnects the stream nor discards the body already received.

Further Reading

• Front-End LLM Streaming Implementation
• Parsing Streaming Structured Data in the Browser
• Web Worker Concurrency
• React Performance Optimization