{"record":{"id":"5eaea8a1f399d60e","repo":"sgl-project/sglang","slug":"lora-pinned-weight-cache-key-collision-for-cache","errorCode":null,"errorMessage":"LoRA pinned weight cache key collision for {cache_key!r}: cached shape={cached_weight.shape}, dtype={cached_weight.dtype}; new shape={weight.shape}, dtype={weight.dtype}.","messagePattern":"LoRA pinned weight cache key collision for (.+?): cached shape=(.+?), dtype=(.+?); new shape=(.+?), dtype=(.+?)\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/lora/mem_pool.py","lineNumber":732,"sourceCode":"        cache_key: str,\n        weight: torch.Tensor,\n    ) -> torch.Tensor:\n        if (\n            not self.pin_memory_available\n            or weight.device.type != \"cpu\"\n            or weight.is_pinned()\n        ):\n            return weight\n\n        if not self.enable_lora_overlap_loading:\n            return weight.pin_memory()\n\n        cached_weight = pinned_weight_store.get(cache_key)\n        if cached_weight is None:\n            cached_weight = weight.pin_memory()\n            pinned_weight_store[cache_key] = cached_weight\n        elif cached_weight.shape != weight.shape or cached_weight.dtype != weight.dtype:\n            raise ValueError(\n                f\"LoRA pinned weight cache key collision for {cache_key!r}: \"\n                f\"cached shape={cached_weight.shape}, dtype={cached_weight.dtype}; \"\n                f\"new shape={weight.shape}, dtype={weight.dtype}.\"\n            )\n\n        return cached_weight\n\n    def prepare_lora_batch(\n        self,\n        cur_uids: Set[Optional[str]],\n        lora_adapters: Dict[str, LoRAAdapter],\n        lora_modules: List[Dict[str, torch.nn.Module]],\n        lora_refs: Dict[str, LoRARef],\n        lora_embed_tokens_module: Optional[BaseLayerWithLoRA],\n        lora_lm_head_module: Optional[BaseLayerWithLoRA],\n    ):\n        # Python hash seeds differ by TP process; slot and LRU updates must not.\n        ordered_uids = sorted(cur_uids, key=lambda uid: (uid is not None, uid or \"\"))","sourceCodeStart":714,"sourceCodeEnd":750,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/lora/mem_pool.py#L714-L750","documentation":"Raised by the LoRA memory pool's pinned-weight transfer cache when the same cache_key maps to a tensor with a different shape or dtype than the cached one. The cache reuses pinned host tensors keyed by (weight identity) to avoid repeated pin_memory allocations; a collision implies two different weights hash to one key, which would silently corrupt transfers.","triggerScenarios":"load_lora_weight_to_buffer calls _get_maybe_cached_weight_for_transfer with a cache_key already present in pinned_weight_store, where the new weight's shape/dtype differs from the cached tensor — e.g. key derived from name only while different adapters/ranks produce different shapes.","commonSituations":"Loading multiple adapters whose weights produce identical cache keys but different dimensions (rank changes between loads), or a key scheme that omits dtype; usually a bug in cache-key construction rather than user config.","solutions":["Make the cache_key include shape and dtype (or rank/config) so distinct weights get distinct keys","Evict the cache entry when adapter configuration changes (clear pinned_weight_store on unload/reconfig)","If you control loading, ensure weights for the same key always come from the same adapter shape/dtype"],"exampleFix":"# before\ncache_key = f\"{lora_name}:{layer}:{weight_name}\"\n# after\ncache_key = f\"{lora_name}:{layer}:{weight_name}:{tuple(weight.shape)}:{weight.dtype}\"","handlingStrategy":"validation","validationCode":"cached = pinned_weight_store.get(cache_key)\nif cached is not None:\n    assert cached.shape == weight.shape and cached.dtype == weight.dtype, f'key collision: {cache_key}'","typeGuard":null,"tryCatchPattern":"try:\n    _get_maybe_cached_weight_for_transfer(weight, cache_key)\nexcept ValueError as e:\n    if 'cache key collision' in str(e):\n        del pinned_weight_store[cache_key]  # evict stale entry, retry\n        _get_maybe_cached_weight_for_transfer(weight, cache_key)\n    else:\n        raise","preventionTips":["Include shape and dtype in cache keys","Clear the pinned-weight cache whenever adapter set or ranks change"],"tags":["lora","cache","shape-mismatch","pin-memory","sglang"],"backgroundTag":"cache-key-collision","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}