Introduction¶

Large Model online inference often batches multiple requests to run them together, making hardware faster, but the numerical path changes with “who is in this batch”: the order of floating-point accumulation and memory layout of the same request can vary, making bit-by-bit output alignment difficult. For auditing, baseline comparison, or downstream stages requiring strict consistency, this is a real-world pain point. The arXiv paper BlockLock: Reproducible LLM Inference with Fixed Batch Geometry (arXiv:2611.06977), submitted on 2026-11-04, discusses: on the premise of keeping shared batch prefilling, only locking in the “geometry” seen by the “protected request,” thereby maintaining bit-consistent output for that request even when peer requests change. Below are the mechanism points and experimental settings organized based on the paper abstract, along with a note on trade-offs with common baselines.

Paper and Release Information¶

The manuscript falls under the category cs.AI with the topic tag cs.CL. The credited authors are Xiuyuan Qi, Pedro Reviriego, Javier Conde, and Shanshan Liu. The version record shows [v1] submitted at UTC 2026-11-04 13:40:05 (Wed, 4 Nov 2026 13:40:05 UTC). The main text entry and DOI can be viewed on the arXiv entry page: https://arxiv.org/abs/2611.06977, and https://doi.org/10.48550/arXiv.2611.06977.

Problem Setting: Peer Changes, Target Unchanged¶

The paper formulates reproducible inference as a hard constraint: the output of a certain request must not change due to changes in other requests within the same batch. The abstract compares two types of existing costs: one is using batch-invariant kernels to constrain every operator; the other is similar to the CoRun baseline, which separates prefilling and uses fixed-shape decoding, at the cost of abandoning shared prefilling computation power and performing padding on the decoding side. BlockLock’s approach is to retain shared batch prefilling and only “nail down” the “geometry” upon which the protected request depends for its numerical path.

What BlockLock’s Two Modes Do¶

First, let’s align on concepts: “Fixed Geometry” refers to the block arrangement, row width, or packing method seen by the protected request in the computation graph being fixed according to deployment contracts, rather than changing all operators in the batch to batch-invariant implementations.

1. BlockLock-Rows: Uses fixed-width rows; the decoding stage can still perform batched decoding for all requests. Under the per-deployment contract mentioned in the abstract, the bit-level output of the target request remains consistent with the contract even when peers change.
2. BlockLock-Pack: Adopts terminal-anchored packing, attaching dummy segments for isolation next to the protected segment, and then extracting that request’s KV state for decoding targeted only at the target. Under the aforementioned contract, changes in peers do not alter the target output bit pattern.

The abstract emphasizes that compared to “batch-invariant kernels for the entire batch,” this approach does not require custom kernel implementations for all requests; the constraint surface is mainly focused on the protected request.

Experiments and Reported Numbers (All from the Paper Abstract)¶

The model side covers five dense models: Qwen3-8B, Qwen3-14B, Llama-3.1-8B-Instruct, Gemma-3-4B-IT, and Gemma-3-12B-IT. The deployment side is evaluated on three single GPU environments: NVIDIA A6000, A100, and H100; initial tensor parallel experiments (two NVIDIA A6000s) are also included. The evaluation data includes tinyBenchmarks and BFCL. The abstract states that in the reported tests, both modes maintain consistency of the target sequence, and the bits of the Top20 logits are consistent with the baseline.

When comparing with vLLM Batch-Invariant and under the condition of “equal delivered work,” the abstract numbers for BlockLock-Rows relative to this baseline are: latency reduced by 53.5%–55.0% and throughput increased by 115.2%–122.2%. The paper also notes that one of its advantages over the batch-invariant kernel route is that it does not require custom kernel implementations.

Implications for Engineering and Scheduling¶

Narrowing reproducibility from “entire model, all batch operators are batch-invariant” to “declaring which requests to protect and their geometry contracts” is operationally closer to “tagging key traffic and writing clear contracts for deployment”; shared prefilling remains, so the trade-off between throughput and latency appears directly in the paper’s numbers—at least for the configuration reported in the abstract, BlockLock-Rows provides a clear range of improvement over the vLLM batch-invariant baseline. Tensor parallelism is only covered in initial experiments; whether it covers larger clusters and more parallel strategies needs to be determined after reading the full text.

If adopted in production, the geometric details, protection set, and interaction with the scheduler within the “per-deployment contract” must still be written into change notes and regression sets; the abstract does not replace a specific implementation manual, but can provide a verifiable common description and data source.

Source: Paper abstract and metadata (submission date, authors, topic classification, etc.) included in the arXiv entry 2611.06977.