sgl-project/sglang · error · ValueError

Parameter {param_name} not found in the model.

Error message

Parameter {param_name} not found in the model.

What it means

While attaching bitsandbytes 4-bit quant states to model parameters, every name in quant_states must exist in the model's params_dict. A quant state with no matching parameter means the checkpoint and model structure disagree.

Source

Thrown at python/sglang/multimodal_gen/runtime/layers/quantization/bitsandbytes.py:314

        state_tensors = {
            name: tensor
            for name, tensor in quant_state_dict.items()
            if name.startswith(f"{source_name}.")
        }
        quant_states[target_name] = QuantState.from_dict(
            state_tensors, device=device_str
        )
    return quant_states


def attach_bitsandbytes_4bit_quant_states(
    params_dict: dict[str, torch.nn.Parameter],
    quant_states: dict[str, Any],
) -> None:
    for param_name, quant_state in quant_states.items():
        param = params_dict.get(param_name)
        if param is None:
            raise ValueError(f"Parameter {param_name} not found in the model.")

        quant_state = _maybe_shard_bitsandbytes_4bit_quant_state(param, quant_state)
        state_by_shard = {0: quant_state}
        set_weight_attrs(param, {"bnb_quant_state": state_by_shard})
        offsets = torch.tensor([0, param.numel()]).cpu()
        set_weight_attrs(param, {"bnb_shard_offsets": offsets})


def _maybe_shard_bitsandbytes_4bit_quant_state(
    param: torch.nn.Parameter,
    quant_state: Any,
) -> Any:
    full_shape = tuple(getattr(param, "bnb_full_shape", tuple(quant_state.shape or ())))
    local_shape = tuple(getattr(param, "bnb_local_shape", full_shape))
    if not full_shape or local_shape == full_shape:
        return quant_state

    output_start = getattr(param, "bnb_output_shard_start", 0)

View on GitHub (pinned to 0132848349)

Solutions

1. Diff quant_states keys against params_dict keys and fix the prefix mapping before attaching
2. Ensure the model was constructed with the same architecture/config as the checkpoint
3. Regenerate the quant state file from the matching checkpoint

missing = set(quant_states) - set(params_dict)
if missing:
    raise SystemExit(f"quant states reference missing params: {sorted(missing)[:5]}")

When it happens

Trigger: attach_bitsandbytes_4bit_quant_states with a quant_states dict containing keys (parameter names) that are absent from params_dict — happens during FSDP loading or bnb 4-bit weight loading when prefix renames differ.

Common situations: Checkpoint saved with different module names than the instantiated model (e.g. wrapped/unwrapped FSDP prefixes, fused module remaps); stale quant state file from another model revision.

Related errors

• SGLang diffusion currently supports AutoRound auto_gptq chec
• AutoRound fused module {target!r} has inconsistent shard con
• The input size is not aligned with the quantized weight shap
• bitsandbytes 4-bit TP only supports column-parallel output s
• bitsandbytes 4-bit TP does not support nested quant states.

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/f01fe6382c5197f9. Report an issue: GitHub.