{"record":{"id":"5946ace4d986bad8","repo":"sgl-project/sglang","slug":"rank-local-fsdp-shard-produced-for-non-dtensor-par","errorCode":null,"errorMessage":"Rank-local FSDP shard produced for non-DTensor parameter {target_param_name}","messagePattern":"Rank-local FSDP shard produced for non-DTensor parameter (.+?)","errorType":"exception","errorClass":"TypeError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/loader/fsdp_load.py","lineNumber":677,"sourceCode":"                    len(quantized_dtype_mismatch_examples[mismatch_key])\n                    < _DTYPE_MISMATCH_EXAMPLE_LIMIT\n                ):\n                    quantized_dtype_mismatch_examples[mismatch_key].append(\n                        target_param_name\n                    )\n            else:\n                non_quantized_dtype_mismatch_counts[mismatch_key] += 1\n                if (\n                    len(non_quantized_dtype_mismatch_examples[mismatch_key])\n                    < _DTYPE_MISMATCH_EXAMPLE_LIMIT\n                ):\n                    non_quantized_dtype_mismatch_examples[mismatch_key].append(\n                        target_param_name\n                    )\n\n        if is_rank_local_fsdp_shard:\n            if not isinstance(meta_sharded_param, dist_tensor.DTensor):\n                raise TypeError(\n                    f\"Rank-local FSDP shard produced for non-DTensor parameter {target_param_name}\"\n                )\n            local_tensor = full_tensor.to(\n                device=checkpoint_load_device,\n                dtype=target_dtype,\n            )\n            sharded_tensor = dist_tensor.DTensor.from_local(\n                local_tensor,\n                meta_sharded_param.device_mesh,\n                meta_sharded_param.placements,\n                run_check=False,\n                shape=meta_sharded_param.shape,\n                stride=meta_sharded_param.stride(),\n            )\n            if cpu_offload:\n                sharded_tensor = sharded_tensor.to(\"cpu\")\n        elif is_rank_local_tp_shard:\n            if isinstance(meta_sharded_param, dist_tensor.DTensor):","sourceCodeStart":659,"sourceCodeEnd":695,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/loader/fsdp_load.py#L659-L695","documentation":"Raised when the FSDP loader takes the rank-local FSDP sharding path for a parameter whose meta-model counterpart is not a torch.distributed.tensor.DTensor. The rank-local FSDP branch assumes the target parameter is FSDP-sharded (a DTensor) so it can derive the local shard; a plain tensor indicates the parameter was never registered for FSDP sharding.","triggerScenarios":"Calling load_model_from_full_model_state_dict (directly or via maybe_load_fsdp_model / _load_weights_into_model / _load_dit_model) where the sharding metadata flags a parameter as is_rank_local_fsdp_shard=True but model.meta_parameters[name] is a plain torch.Tensor rather than a DTensor — e.g. a module wrapped incorrectly or excluded from the FSDP wrap policy.","commonSituations":"Changing the FSDP wrapping policy or auto_wrap hints so some parameters stop being FSDP-sharded while the loader's sharding bookkeeping still marks them rank-local; mixing FSDP and non-FSDP submodules; upgrading torch or model code where a param class no longer produces DTensors on meta init.","solutions":["Check how is_rank_local_fsdp_shard was computed for this parameter and why meta_sharded_param is not a DTensor (print type(model parameter) on meta init)","Fix the FSDP wrap policy / sharding metadata so the parameter is actually FSDP-sharded (DTensor) or routed to the plain-tensor path","Ensure the model is meta-initialized under the same FSDP configuration used at load time"],"exampleFix":"# before: param excluded from FSDP wrap, loader treats it as rank-local shard\n# after: include the module in the FSDP wrap policy so its params become DTensors\nauto_wrap_policy = functools.partial(\n    size_based_auto_wrap_policy, min_num_params=int(1e6)\n)  # ensure the module holding target_param_name is wrapped","handlingStrategy":"validation","validationCode":"import torch.distributed.tensor as dist_tensor\nfor name, p in model.named_parameters():\n    if sharding_plan[name].is_rank_local_fsdp_shard:\n        assert isinstance(p, dist_tensor.DTensor), f\"{name} not FSDP-sharded\"","typeGuard":"def is_fsdp_dtensor(p) -> bool:\n    import torch.distributed.tensor as dist_tensor\n    return isinstance(p, dist_tensor.DTensor)","tryCatchPattern":"try:\n    load_model_from_full_model_state_dict(...)\nexcept TypeError as e:\n    if 'Rank-local FSDP shard' in str(e):\n        fix_wrap_policy(); retry_load()","preventionTips":["Keep FSDP wrap policy identical between meta init and load","Add a startup assert that all rank-local FSDP params are DTensors","Log the sharding classification per parameter in debug builds"],"tags":["fsdp","dtensor","distributed","weight-loading","sharding"],"backgroundTag":"fsdp-sharding-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}