{"record":{"id":"eea4cc91866d8bd5","repo":"sgl-project/sglang","slug":"shard-offset-and-shard-size-must-be-provided","errorCode":null,"errorMessage":"shard_offset and shard_size must be provided","messagePattern":"shard_offset and shard_size must be provided","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/models/parameter.py","lineNumber":118,"sourceCode":"    @property\n    def output_dim(self):\n        return self._output_dim\n\n    def load_column_parallel_weight(self, loaded_weight: torch.Tensor) -> None:\n        tp_rank = get_tp_rank()\n        shard_size = self.data.shape[self.output_dim]\n        loaded_weight = loaded_weight.narrow(\n            self.output_dim, tp_rank * shard_size, shard_size\n        )\n        assert self.data.shape == loaded_weight.shape\n        self.data.copy_(loaded_weight)\n\n    def load_merged_column_weight(self, loaded_weight: torch.Tensor, **kwargs) -> None:\n\n        shard_offset = kwargs.get(\"shard_offset\")\n        shard_size = kwargs.get(\"shard_size\")\n        if shard_offset is None or shard_size is None:\n            raise ValueError(\"shard_offset and shard_size must be provided\")\n        if (\n            isinstance(self, PackedColumnParameter | PackedvLLMParameter)\n            and self.packed_dim == self.output_dim\n        ):\n            shard_size, shard_offset = self.adjust_shard_indexes_for_packing(\n                shard_offset=shard_offset, shard_size=shard_size\n            )\n\n        param_data = self.data\n\n        tp_rank = get_tp_rank()\n        param_data = param_data.narrow(self.output_dim, shard_offset, shard_size)\n        loaded_weight = loaded_weight.narrow(\n            self.output_dim, tp_rank * shard_size, shard_size\n        )\n        assert param_data.shape == loaded_weight.shape\n        param_data.copy_(loaded_weight)\n","sourceCodeStart":100,"sourceCodeEnd":136,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/models/parameter.py#L100-L136","documentation":"load_merged_column_weight requires shard_offset and shard_size so it knows where within the merged column-parallel weight to place the loaded shard. Without them the write location is undefined, so the API raises ValueError immediately.","triggerScenarios":"Calling parameter.load_merged_column_weight(loaded_weight) without passing shard_offset/shard_size kwargs — typically when a weight loader for a non-sharded (fully loaded) weight accidentally routes to the merged-column path, or a custom loader forgets the kwargs.","commonSituations":"Porting a vLLM-style model implementation whose weight_mapping omits shard metadata; checkpoints where a gate/up projection is stored as one fused tensor and the loader takes the 'no shard' branch; refactors of weight_loading_utils that drop kwargs.","solutions":["Pass shard_offset and shard_size describing where the shard sits in the merged dimension (e.g. shard_offset=0, shard_size=num_kv_heads*head_dim for the first shard)","If the tensor is not actually column-sharded, route it to load_weight / the plain Parameter loader instead of load_merged_column_weight","For PackedColumnParameter with packed_dim == output_dim, confirm the offsets are in unpacked units — the method will adjust them for packing"],"exampleFix":"// before\nparam.load_merged_column_weight(w)\n// after\nparam.load_merged_column_weight(w, shard_offset=shard_id * shard_size, shard_size=shard_size)","handlingStrategy":"validation","validationCode":"if 'shard_offset' not in kwargs or 'shard_size' not in kwargs:\n    param.load_weight(loaded_weight)  # non-merged path\nelse:\n    param.load_merged_column_weight(loaded_weight, **kwargs)","typeGuard":"def is_column_sharded(name: str, weights: dict) -> bool:\n    return name in weights and isinstance(weights[name], tuple)  # (offset, size) entries","tryCatchPattern":"try:\n    param.load_merged_column_weight(w, shard_offset=off, shard_size=sz)\nexcept ValueError as e:\n    logger.error(\"missing shard metadata for %s\", param.name); raise","preventionTips":["Always compute shard_offset/shard_size in the weight-mapping loop before dispatching to merged loaders","Only route genuinely column-split tensors to load_merged_column_weight","Unit-test the loader on a tiny checkpoint covering both fused and split tensors"],"tags":["weight-loading","column-parallel","shard","checkpoint"],"backgroundTag":"missing-required-argument","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}