sgl-project/sglang · error · ValueError

Cannot resolve deferred scale_inv {scale_name}: weight {weig

Error message

Cannot resolve deferred scale_inv {scale_name}: weight {weight_name} not found

What it means

During deferred scale_inv resolution, the corresponding .weight parameter derived by replacing '.weight_scale_inv' with '.weight' is not present in params_dict. The deferred scale was captured but has no matching parameter to size against.

Source

Thrown at python/sglang/srt/models/mimo_v2.py:203

    return torch.cat(all_q + all_k + all_v, dim=0)


def _resolve_deferred_qkv_scale_inv(
    params_dict: Dict[str, torch.nn.Parameter],
    deferred_scale_inv: Dict[str, torch.Tensor],
    expected_fused_tp_size: int,
    block_size: int = 128,
    config=None,
):
    tp_size = get_parallel().attn_tp_size
    tp_rank = get_parallel().attn_tp_rank
    ckpt_tp = expected_fused_tp_size
    shards_per_rank = ckpt_tp // tp_size

    for scale_name, ckpt_scale in deferred_scale_inv.items():
        weight_name = scale_name.replace(".weight_scale_inv", ".weight")
        if weight_name not in params_dict:
            raise ValueError(
                f"Cannot resolve deferred scale_inv {scale_name}: "
                f"weight {weight_name} not found"
            )

        weight_param = params_dict[weight_name]
        scale_param = params_dict[scale_name]
        weight_data = weight_param.data

        ckpt_scale_shards = ckpt_scale.chunk(ckpt_tp, dim=0)
        my_scale_shards = ckpt_scale_shards[
            tp_rank * shards_per_rank : (tp_rank + 1) * shards_per_rank
        ]

        weight_rows = weight_data.shape[0]
        rows_per_ckpt_shard = weight_rows // shards_per_rank
        block_k = ckpt_scale.shape[1]

        device = weight_data.device

View on GitHub (pinned to 0132848349)

Solutions

1. Ensure every weight_scale_inv tensor has a matching .weight tensor with the same name in the checkpoint
2. Verify parameter names in the converted checkpoint match the model's params_dict keys (qkv_proj naming)
3. Drop orphan scale_inv tensors during conversion if their weights are intentionally absent

missing = [s for s in deferred if s.replace('.weight_scale_inv', '.weight') not in params_dict]
if missing:
    raise SystemExit(f'orphan scale_inv tensors: {missing}')

def scale_has_weight(scale_name: str, params_dict: dict) -> bool:
    return scale_name.replace('.weight_scale_inv', '.weight') in params_dict

When it happens

Trigger: A scale_name stored in deferred_scale_inv whose sibling weight tensor is absent from params_dict — either the weight was skipped/renamed, or the param mapping (name replace) doesn't match the model's parameter naming.

Common situations: Custom quantized checkpoint names that don't follow 'model.layers.X.self_attn.qkv_proj.weight[_scale_inv]' convention; partial checkpoints; renaming during conversion.

Related errors

• qkv_proj scale_inv {name}: shape mismatch {tuple(loaded_weig
• Pack: Only supports tensors with dimensions not greater than
• Expected scalar scale for fused-in-checkpoint merged-column
• Block-quantized lm_head is not supported; use channel or ten
• Eagle3 MLA draft post_load_weights only supports float dtype

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/46f2e5227fc2b21f. Report an issue: GitHub.