{"record":{"id":"f01fe6382c5197f9","repo":"sgl-project/sglang","slug":"parameter-param-name-not-found-in-the-model","errorCode":null,"errorMessage":"Parameter {param_name} not found in the model.","messagePattern":"Parameter (.+?) not found in the model\\.","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/layers/quantization/bitsandbytes.py","lineNumber":314,"sourceCode":"        state_tensors = {\n            name: tensor\n            for name, tensor in quant_state_dict.items()\n            if name.startswith(f\"{source_name}.\")\n        }\n        quant_states[target_name] = QuantState.from_dict(\n            state_tensors, device=device_str\n        )\n    return quant_states\n\n\ndef attach_bitsandbytes_4bit_quant_states(\n    params_dict: dict[str, torch.nn.Parameter],\n    quant_states: dict[str, Any],\n) -> None:\n    for param_name, quant_state in quant_states.items():\n        param = params_dict.get(param_name)\n        if param is None:\n            raise ValueError(f\"Parameter {param_name} not found in the model.\")\n\n        quant_state = _maybe_shard_bitsandbytes_4bit_quant_state(param, quant_state)\n        state_by_shard = {0: quant_state}\n        set_weight_attrs(param, {\"bnb_quant_state\": state_by_shard})\n        offsets = torch.tensor([0, param.numel()]).cpu()\n        set_weight_attrs(param, {\"bnb_shard_offsets\": offsets})\n\n\ndef _maybe_shard_bitsandbytes_4bit_quant_state(\n    param: torch.nn.Parameter,\n    quant_state: Any,\n) -> Any:\n    full_shape = tuple(getattr(param, \"bnb_full_shape\", tuple(quant_state.shape or ())))\n    local_shape = tuple(getattr(param, \"bnb_local_shape\", full_shape))\n    if not full_shape or local_shape == full_shape:\n        return quant_state\n\n    output_start = getattr(param, \"bnb_output_shard_start\", 0)","sourceCodeStart":296,"sourceCodeEnd":332,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/layers/quantization/bitsandbytes.py#L296-L332","documentation":"While attaching bitsandbytes 4-bit quant states to model parameters, every name in quant_states must exist in the model's params_dict. A quant state with no matching parameter means the checkpoint and model structure disagree.","triggerScenarios":"attach_bitsandbytes_4bit_quant_states with a quant_states dict containing keys (parameter names) that are absent from params_dict — happens during FSDP loading or bnb 4-bit weight loading when prefix renames differ.","commonSituations":"Checkpoint saved with different module names than the instantiated model (e.g. wrapped/unwrapped FSDP prefixes, fused module remaps); stale quant state file from another model revision.","solutions":["Diff quant_states keys against params_dict keys and fix the prefix mapping before attaching","Ensure the model was constructed with the same architecture/config as the checkpoint","Regenerate the quant state file from the matching checkpoint"],"exampleFix":null,"handlingStrategy":"validation","validationCode":"missing = set(quant_states) - set(params_dict)\nif missing:\n    raise SystemExit(f\"quant states reference missing params: {sorted(missing)[:5]}\")","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Diff quant_state keys against model state_dict keys before attaching","Keep checkpoint and model prefix naming consistent"],"tags":["quantization","bitsandbytes","parameter-mapping","checkpoint"],"backgroundTag":"checkpoint-parameter-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}