{"record":{"id":"1fee3476547d5308","repo":"vllm-project/vllm","slug":"partial-tail-offloads-for-one-request-must-share-a","errorCode":null,"errorMessage":"Partial-tail offloads for one request must share a boundary","messagePattern":"Partial-tail offloads for one request must share a boundary","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"vllm/distributed/kv_transfer/kv_connector/v1/mooncake/store/worker.py","lineNumber":589,"sourceCode":"        the normal save floors to ``lcm_block_size``, so a smaller-block\n        group's full blocks in that gap are never persisted elsewhere, and\n        the consumer's lookup needs every group at every probed boundary.\n        Full blocks are keyed by their block-end hash, the partial boundary\n        block by the boundary sub-hash; the mamba \"align\" boundary block is\n        the core-provided CoW block. All keys are deduped against the store.\n\n        Returns:\n            True when no put is needed or every put succeeds, False otherwise.\n        \"\"\"\n        if not self.coord.enable_partial_hash_hits or not req_meta.block_hashes:\n            return True\n        partial_tail_offloads = req_meta.partial_tail_offloads\n        if not partial_tail_offloads:\n            return True\n        hash_block_size = self.coord.hash_block_size\n        boundaries = {boundary for _, _, boundary in partial_tail_offloads}\n        if len(boundaries) != 1:\n            raise ValueError(\n                \"Partial-tail offloads for one request must share a boundary\"\n            )\n        boundary = boundaries.pop()\n        if boundary == 0:\n            return True\n        if boundary // hash_block_size - 1 >= len(req_meta.block_hashes):\n            return True\n        mamba_offloads = {\n            group_id: block_id for group_id, block_id, _ in partial_tail_offloads\n        }\n\n        keys: list[str] = []\n        addrs: list[list[int]] = []\n        sizes: list[list[int]] = []\n        group_ids: list[str] | None = (\n            [] if self.enable_group_semantics and self.supports_group_ids else None\n        )\n        saved = self._saved_offset.get(req_meta.req_id, 0)","sourceCodeStart":571,"sourceCodeEnd":607,"githubUrl":"https://github.com/vllm-project/vllm/blob/c794754062d49a8fdb63ab3c5215b488b865030c/vllm/distributed/kv_transfer/kv_connector/v1/mooncake/store/worker.py#L571-L607","documentation":"When enable_partial_hash_hits is on, the worker offloads the unaligned tail of a request in hash_block_size chunks; it collects (group_id, block_id, boundary) partial-tail offload tuples and asserts every group in one request reports the same boundary token count. Divergent boundaries across KV cache groups would slice the same request inconsistently, so it raises ValueError before doing any store put.","triggerScenarios":"A hybrid/multi-group model where group A's partial tail boundary differs from group B's for the same request — typically caused by hash_block_size interacting differently with per-group block sizes, or a scheduler/connector invariant break.","commonSituations":"Enabling kv_connector_extra_config['enable_partial_hash_hits']=true on hybrid models; custom hash_block_size values on multi-group caches; version skew between scheduler metadata and connector expectations.","solutions":["Disable 'enable_partial_hash_hits' in kv_connector_extra_config (partial tail offload becomes all-or-nothing)","Ensure per-group block sizes align (mamba_cache_mode='align') so boundaries cannot diverge","If configuration is consistent and it still fires, capture scheduler_output + req_meta for the request and file a vLLM issue — it is an internal invariant"],"exampleFix":"# before\nkv_connector_extra_config={\"enable_partial_hash_hits\": True}\n\n# after\nkv_connector_extra_config={}  # or enable_partial_hash_hits: False","handlingStrategy":"fallback","validationCode":"# feature-detect before enabling the advanced mode\nif is_multi_group(vllm_config) and extra_config.get(\"enable_partial_hash_hits\"):\n    logger.warning(\"partial hash hits on multi-group caches is fragile; disabling\")\n    extra_config = {**extra_config, \"enable_partial_hash_hits\": False}","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Leave enable_partial_hash_hits off unless you need partial-tail offload","Keep per-group block sizes aligned (mamba_cache_mode='align') on hybrid models","Pin connector + scheduler to a tested vLLM release"],"tags":["mooncake","kv-transfer","internal-invariant","hybrid-attention"],"backgroundTag":null,"analyzedSha":"c794754062d49a8fdb63ab3c5215b488b865030c","analyzedAt":"2026-08-14T21:17:39.825Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}