sgl-project/sglang · error · TypeError

Rank-local TP shard produced for DTensor parameter {target_p

Error message

Rank-local TP shard produced for DTensor parameter {target_param_name}

What it means

Raised when the loader takes the rank-local TP (tensor-parallel) sharding path but the target parameter in the meta model is a DTensor. The TP path expects plain (FSDP-unsharded) tensors it can slice with weight_loader; a DTensor means the parameter is simultaneously marked for both TP and FSDP rank-local handling, which is unsupported.

Source

Thrown at python/sglang/multimodal_gen/runtime/loader/fsdp_load.py:696

                    f"Rank-local FSDP shard produced for non-DTensor parameter {target_param_name}"
                )
            local_tensor = full_tensor.to(
                device=checkpoint_load_device,
                dtype=target_dtype,
            )
            sharded_tensor = dist_tensor.DTensor.from_local(
                local_tensor,
                meta_sharded_param.device_mesh,
                meta_sharded_param.placements,
                run_check=False,
                shape=meta_sharded_param.shape,
                stride=meta_sharded_param.stride(),
            )
            if cpu_offload:
                sharded_tensor = sharded_tensor.to("cpu")
        elif is_rank_local_tp_shard:
            if isinstance(meta_sharded_param, dist_tensor.DTensor):
                raise TypeError(
                    f"Rank-local TP shard produced for DTensor parameter {target_param_name}"
                )
            sharded_tensor = full_tensor.to(
                device=checkpoint_load_device,
                dtype=target_dtype,
            )
            if cpu_offload:
                sharded_tensor = sharded_tensor.cpu()
        elif not isinstance(meta_sharded_param, dist_tensor.DTensor):
            full_tensor = full_tensor.to(
                device=checkpoint_load_device,
                dtype=target_dtype,
            )
            actual_param = rank_local_checkpoint.get_param_for_weight_loading(
                model, param_dict, target_param_name
            )
            weight_loader = (
                getattr(actual_param, "weight_loader", None)

View on GitHub (pinned to 0132848349)

Solutions

1. Inspect how the parameter was classified (TP rank-local vs FSDP) and why meta init produced a DTensor for it
2. Make the meta model and the loader agree: either the param is FSDP-sharded (DTensor, FSDP path) or a plain tensor TP-sharded (TP path)
3. Rebuild the meta model with the same parallelism configuration used to build the loading plan

import torch.distributed.tensor as dist_tensor
for name, p in model.named_parameters():
    if sharding_plan[name].is_rank_local_tp_shard:
        assert not isinstance(p, dist_tensor.DTensor), f"{name} is DTensor but TP path chosen"

def is_plain_tensor(p) -> bool:
    import torch.distributed.tensor as dist_tensor
    return not isinstance(p, dist_tensor.DTensor)

try:
    load_model_from_full_model_state_dict(...)
except TypeError as e:
    if 'Rank-local TP shard' in str(e):
        align_parallelism_config(); retry_load()

When it happens

Trigger: load_model_from_full_model_state_dict with is_rank_local_tp_shard=True while meta_sharded_param is a dist_tensor.DTensor — typically when a model is FSDP-wrapped but the loader/classifier decided the parameter should be TP-sharded as a rank-local plain tensor.

Common situations: Switching a checkpoint or model between pure-TP and FSDP+TP hybrid execution; a weight_loader or param classification bug that marks FSDP DTensor params as TP rank-local; version drift between the meta-model construction and loader sharding metadata.

Related errors

• Rank-local FSDP shard produced for non-DTensor parameter {ta
• GGUF diffusion checkpoints are incompatible with FSDP infere
• Invalid {tp_size=}. Expected tp_size >= 1.
• Failed to load MiniMax H3 Qwen3-VL weight {name!r}: checkpoi
• MiMoV2 fused qkv_proj checkpoint is TP={expected_fused_tp_si

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/211843f5f8813564. Report an issue: GitHub.