{"record":{"id":"211843f5f8813564","repo":"sgl-project/sglang","slug":"rank-local-tp-shard-produced-for-dtensor-parameter","errorCode":null,"errorMessage":"Rank-local TP shard produced for DTensor parameter {target_param_name}","messagePattern":"Rank-local TP shard produced for DTensor parameter (.+?)","errorType":"exception","errorClass":"TypeError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/loader/fsdp_load.py","lineNumber":696,"sourceCode":"                    f\"Rank-local FSDP shard produced for non-DTensor parameter {target_param_name}\"\n                )\n            local_tensor = full_tensor.to(\n                device=checkpoint_load_device,\n                dtype=target_dtype,\n            )\n            sharded_tensor = dist_tensor.DTensor.from_local(\n                local_tensor,\n                meta_sharded_param.device_mesh,\n                meta_sharded_param.placements,\n                run_check=False,\n                shape=meta_sharded_param.shape,\n                stride=meta_sharded_param.stride(),\n            )\n            if cpu_offload:\n                sharded_tensor = sharded_tensor.to(\"cpu\")\n        elif is_rank_local_tp_shard:\n            if isinstance(meta_sharded_param, dist_tensor.DTensor):\n                raise TypeError(\n                    f\"Rank-local TP shard produced for DTensor parameter {target_param_name}\"\n                )\n            sharded_tensor = full_tensor.to(\n                device=checkpoint_load_device,\n                dtype=target_dtype,\n            )\n            if cpu_offload:\n                sharded_tensor = sharded_tensor.cpu()\n        elif not isinstance(meta_sharded_param, dist_tensor.DTensor):\n            full_tensor = full_tensor.to(\n                device=checkpoint_load_device,\n                dtype=target_dtype,\n            )\n            actual_param = rank_local_checkpoint.get_param_for_weight_loading(\n                model, param_dict, target_param_name\n            )\n            weight_loader = (\n                getattr(actual_param, \"weight_loader\", None)","sourceCodeStart":678,"sourceCodeEnd":714,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/loader/fsdp_load.py#L678-L714","documentation":"Raised when the loader takes the rank-local TP (tensor-parallel) sharding path but the target parameter in the meta model is a DTensor. The TP path expects plain (FSDP-unsharded) tensors it can slice with weight_loader; a DTensor means the parameter is simultaneously marked for both TP and FSDP rank-local handling, which is unsupported.","triggerScenarios":"load_model_from_full_model_state_dict with is_rank_local_tp_shard=True while meta_sharded_param is a dist_tensor.DTensor — typically when a model is FSDP-wrapped but the loader/classifier decided the parameter should be TP-sharded as a rank-local plain tensor.","commonSituations":"Switching a checkpoint or model between pure-TP and FSDP+TP hybrid execution; a weight_loader or param classification bug that marks FSDP DTensor params as TP rank-local; version drift between the meta-model construction and loader sharding metadata.","solutions":["Inspect how the parameter was classified (TP rank-local vs FSDP) and why meta init produced a DTensor for it","Make the meta model and the loader agree: either the param is FSDP-sharded (DTensor, FSDP path) or a plain tensor TP-sharded (TP path)","Rebuild the meta model with the same parallelism configuration used to build the loading plan"],"exampleFix":null,"handlingStrategy":"validation","validationCode":"import torch.distributed.tensor as dist_tensor\nfor name, p in model.named_parameters():\n    if sharding_plan[name].is_rank_local_tp_shard:\n        assert not isinstance(p, dist_tensor.DTensor), f\"{name} is DTensor but TP path chosen\"","typeGuard":"def is_plain_tensor(p) -> bool:\n    import torch.distributed.tensor as dist_tensor\n    return not isinstance(p, dist_tensor.DTensor)","tryCatchPattern":"try:\n    load_model_from_full_model_state_dict(...)\nexcept TypeError as e:\n    if 'Rank-local TP shard' in str(e):\n        align_parallelism_config(); retry_load()","preventionTips":["Ensure FSDP and TP classification of each parameter is exclusive","Test loads with the exact parallelism config used in production","Fail fast on config changes by asserting parameter types on meta model"],"tags":["tensor-parallel","fsdp","dtensor","distributed","weight-loading"],"backgroundTag":"parallelism-config-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}