sgl-project/sglang · error · ValueError

LoRA pinned weight cache key collision for {cache_key!r}: ca

Error message

LoRA pinned weight cache key collision for {cache_key!r}: cached shape={cached_weight.shape}, dtype={cached_weight.dtype}; new shape={weight.shape}, dtype={weight.dtype}.

What it means

Raised by the LoRA memory pool's pinned-weight transfer cache when the same cache_key maps to a tensor with a different shape or dtype than the cached one. The cache reuses pinned host tensors keyed by (weight identity) to avoid repeated pin_memory allocations; a collision implies two different weights hash to one key, which would silently corrupt transfers.

Source

Thrown at python/sglang/srt/lora/mem_pool.py:732

        cache_key: str,
        weight: torch.Tensor,
    ) -> torch.Tensor:
        if (
            not self.pin_memory_available
            or weight.device.type != "cpu"
            or weight.is_pinned()
        ):
            return weight

        if not self.enable_lora_overlap_loading:
            return weight.pin_memory()

        cached_weight = pinned_weight_store.get(cache_key)
        if cached_weight is None:
            cached_weight = weight.pin_memory()
            pinned_weight_store[cache_key] = cached_weight
        elif cached_weight.shape != weight.shape or cached_weight.dtype != weight.dtype:
            raise ValueError(
                f"LoRA pinned weight cache key collision for {cache_key!r}: "
                f"cached shape={cached_weight.shape}, dtype={cached_weight.dtype}; "
                f"new shape={weight.shape}, dtype={weight.dtype}."
            )

        return cached_weight

    def prepare_lora_batch(
        self,
        cur_uids: Set[Optional[str]],
        lora_adapters: Dict[str, LoRAAdapter],
        lora_modules: List[Dict[str, torch.nn.Module]],
        lora_refs: Dict[str, LoRARef],
        lora_embed_tokens_module: Optional[BaseLayerWithLoRA],
        lora_lm_head_module: Optional[BaseLayerWithLoRA],
    ):
        # Python hash seeds differ by TP process; slot and LRU updates must not.
        ordered_uids = sorted(cur_uids, key=lambda uid: (uid is not None, uid or ""))

View on GitHub (pinned to 0132848349)

Solutions

1. Make the cache_key include shape and dtype (or rank/config) so distinct weights get distinct keys
2. Evict the cache entry when adapter configuration changes (clear pinned_weight_store on unload/reconfig)
3. If you control loading, ensure weights for the same key always come from the same adapter shape/dtype

Example fix

# before
cache_key = f"{lora_name}:{layer}:{weight_name}"
# after
cache_key = f"{lora_name}:{layer}:{weight_name}:{tuple(weight.shape)}:{weight.dtype}"

cached = pinned_weight_store.get(cache_key)
if cached is not None:
    assert cached.shape == weight.shape and cached.dtype == weight.dtype, f'key collision: {cache_key}'

try:
    _get_maybe_cached_weight_for_transfer(weight, cache_key)
except ValueError as e:
    if 'cache key collision' in str(e):
        del pinned_weight_store[cache_key]  # evict stale entry, retry
        _get_maybe_cached_weight_for_transfer(weight, cache_key)
    else:
        raise

When it happens

Trigger: load_lora_weight_to_buffer calls _get_maybe_cached_weight_for_transfer with a cache_key already present in pinned_weight_store, where the new weight's shape/dtype differs from the cached tensor — e.g. key derived from name only while different adapters/ranks produce different shapes.

Common situations: Loading multiple adapters whose weights produce identical cache keys but different dimensions (rank changes between loads), or a key scheme that omits dtype; usually a bug in cache-key construction rather than user config.

Related errors

• Mixed shared-outer LoRA formats detected across loaded adapt
• chunk_plucker shape mismatch for SANA-WM: expected {expected
• Resolved LoRA weight {selected_file!r} was not downloaded to
• DSA indexer weights_proj LoRA is incompatible with piecewise
• Failed to load LoRA adapter {lora_ref.lora_name} because it

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/5eaea8a1f399d60e. Report an issue: GitHub.