vllm-project/vllm · error · ValueError

Partial-tail offloads for one request must share a boundary

Error message

Partial-tail offloads for one request must share a boundary

What it means

When enable_partial_hash_hits is on, the worker offloads the unaligned tail of a request in hash_block_size chunks; it collects (group_id, block_id, boundary) partial-tail offload tuples and asserts every group in one request reports the same boundary token count. Divergent boundaries across KV cache groups would slice the same request inconsistently, so it raises ValueError before doing any store put.

Source

Thrown at vllm/distributed/kv_transfer/kv_connector/v1/mooncake/store/worker.py:589

        the normal save floors to ``lcm_block_size``, so a smaller-block
        group's full blocks in that gap are never persisted elsewhere, and
        the consumer's lookup needs every group at every probed boundary.
        Full blocks are keyed by their block-end hash, the partial boundary
        block by the boundary sub-hash; the mamba "align" boundary block is
        the core-provided CoW block. All keys are deduped against the store.

        Returns:
            True when no put is needed or every put succeeds, False otherwise.
        """
        if not self.coord.enable_partial_hash_hits or not req_meta.block_hashes:
            return True
        partial_tail_offloads = req_meta.partial_tail_offloads
        if not partial_tail_offloads:
            return True
        hash_block_size = self.coord.hash_block_size
        boundaries = {boundary for _, _, boundary in partial_tail_offloads}
        if len(boundaries) != 1:
            raise ValueError(
                "Partial-tail offloads for one request must share a boundary"
            )
        boundary = boundaries.pop()
        if boundary == 0:
            return True
        if boundary // hash_block_size - 1 >= len(req_meta.block_hashes):
            return True
        mamba_offloads = {
            group_id: block_id for group_id, block_id, _ in partial_tail_offloads
        }

        keys: list[str] = []
        addrs: list[list[int]] = []
        sizes: list[list[int]] = []
        group_ids: list[str] | None = (
            [] if self.enable_group_semantics and self.supports_group_ids else None
        )
        saved = self._saved_offset.get(req_meta.req_id, 0)

View on GitHub (pinned to c794754062)

Solutions

1. Disable 'enable_partial_hash_hits' in kv_connector_extra_config (partial tail offload becomes all-or-nothing)
2. Ensure per-group block sizes align (mamba_cache_mode='align') so boundaries cannot diverge
3. If configuration is consistent and it still fires, capture scheduler_output + req_meta for the request and file a vLLM issue — it is an internal invariant

Example fix

# before
kv_connector_extra_config={"enable_partial_hash_hits": True}

# after
kv_connector_extra_config={}  # or enable_partial_hash_hits: False

# feature-detect before enabling the advanced mode
if is_multi_group(vllm_config) and extra_config.get("enable_partial_hash_hits"):
    logger.warning("partial hash hits on multi-group caches is fragile; disabling")
    extra_config = {**extra_config, "enable_partial_hash_hits": False}

When it happens

Trigger: A hybrid/multi-group model where group A's partial tail boundary differs from group B's for the same request — typically caused by hash_block_size interacting differently with per-group block sizes, or a scheduler/connector invariant break.

Common situations: Enabling kv_connector_extra_config['enable_partial_hash_hits']=true on hybrid models; custom hash_block_size values on multi-group caches; version skew between scheduler metadata and connector expectations.

Related errors

• MooncakeStoreConnector does not support: {unsupported}
• Group count mismatch: tracker has {len(self.allocated_block_
• Mooncake is not available
• Mooncake Transfer Engine initialization failed.
• No KV cache tensors were registered with Mooncake.

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/1fee3476547d5308. Report an issue: GitHub.