{"record":{"id":"c02a2cbcc41152cf","repo":"sgl-project/sglang","slug":"shard-id-with-multiple-indices-is-not-supported-in","errorCode":null,"errorMessage":"Shard id with multiple indices is not supported in weight_loader, please use weight_loader_v2 instead.","messagePattern":"Shard id with multiple indices is not supported in weight_loader, please use weight_loader_v2 instead\\.","errorType":"validation","errorClass":"NotImplementedError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/layers/linear.py","lineNumber":585,"sourceCode":"            params_dtype=params_dtype,\n            quant_config=quant_config,\n            prefix=prefix,\n            tp_rank=tp_rank,\n            tp_size=tp_size,\n            use_presharded_weights=use_presharded_weights,\n        )\n        self.prefix = prefix\n\n    def weight_loader(\n        self,\n        param: Parameter,\n        loaded_weight: torch.Tensor,\n        loaded_shard_id: tuple[int, ...] | int | None = None,\n    ):\n        if isinstance(loaded_shard_id, tuple):\n            if hasattr(param, \"load_merged_column_weight\"):\n                return self.weight_loader_v2(param, loaded_weight, loaded_shard_id)\n            raise NotImplementedError(\n                \"Shard id with multiple indices is not supported in weight_loader, \"\n                \"please use weight_loader_v2 instead.\"\n            )\n\n        # Special case for GGUF\n        # initialize GGUF param after we know the quantize type\n        is_gguf_weight = getattr(param, \"is_gguf_weight\", False)\n        is_gguf_weight_type = getattr(param, \"is_gguf_weight_type\", False)\n        if is_gguf_weight_type:\n            param.data[loaded_shard_id].copy_(loaded_weight)\n            param.shard_weight_type[loaded_shard_id] = loaded_weight.item()\n            return\n\n        if is_gguf_weight:\n            output_dim = getattr(param, \"output_dim\", None)\n            shard_size = loaded_weight.size(output_dim) // self.tp_size\n            start_idx = self.tp_rank * shard_size\n","sourceCodeStart":567,"sourceCodeEnd":603,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/layers/linear.py#L567-L603","documentation":"The v1 weight_loader cannot handle a tuple loaded_shard_id (a shard spanning multiple indices, e.g. fused or nested shards). If the param lacks a load_merged_column_weight hook to redirect to weight_loader_v2, it raises NotImplementedError telling you to use weight_loader_v2.","triggerScenarios":"Calling weight_loader(param, loaded_weight, loaded_shard_id=(i, j)) on a linear whose parameter is not a merged-column param (no load_merged_column_weight attribute) — e.g. custom quant methods or model code invoking the v1 loader with multi-index shard ids.","commonSituations":"Custom quantization or model classes overriding/forwarding to v1 weight_loader while the checkpoint uses fused shards; new model implementations passing tuple shard ids to the old loader API.","solutions":["Call weight_loader_v2 instead, which natively supports tuple shard ids","Make the param a merged-column parameter (attach load_merged_column_weight) so v1 auto-delegates","Update the model/quant code to stop constructing tuple shard ids for the v1 loader"],"exampleFix":"# before\nlinear.weight_loader(param, loaded_weight, loaded_shard_id=(1, 2))  # NotImplementedError\n\n# after\nlinear.weight_loader_v2(param, loaded_weight, loaded_shard_id=(1, 2))","handlingStrategy":"type-guard","validationCode":"if isinstance(loaded_shard_id, tuple):\n    assert hasattr(param, \"load_merged_column_weight\"), \"v1 loader cannot take tuple shard ids\"\n    linear.weight_loader_v2(param, loaded_weight, loaded_shard_id)\nelse:\n    linear.weight_loader(param, loaded_weight, loaded_shard_id)","typeGuard":"def needs_v2_loader(param, loaded_shard_id) -> bool:\n    return isinstance(loaded_shard_id, tuple) and not hasattr(param, \"load_merged_column_weight\")","tryCatchPattern":"try:\n    linear.weight_loader(param, loaded_weight, loaded_shard_id)\nexcept NotImplementedError as e:\n    if \"weight_loader_v2\" in str(e):\n        linear.weight_loader_v2(param, loaded_weight, loaded_shard_id)\n    else:\n        raise","preventionTips":["Route tuple shard ids straight to weight_loader_v2","Attach load_merged_column_weight to merged params so v1 auto-delegates","Avoid constructing multi-index shard ids in new model code targeting v1"],"tags":["weight-loading","shard-id","merged-column","api-version"],"backgroundTag":"unsupported-operation-legacy-api","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}