sgl-project/sglang · error · ValueError

Unsupported dtype in flattened_bucket metadata: {dtype!r}

Error message

Unsupported dtype in flattened_bucket metadata: {dtype!r}

What it means

Raised by WeightsUpdater._normalize_torch_dtype when a dtype in flattened_bucket metadata cannot be resolved to a torch.dtype. Strings like 'torch.float32', 'float32', or 'bfloat16' work via getattr on the last dotted component; anything else (integers, unknown names) fails.

Source

Thrown at python/sglang/multimodal_gen/runtime/post_training/weights_updater.py:823

                    numel=int(meta.numel),
                )
            )

        bucket = FlattenedTensorBucket(
            flattened_tensor=flattened_tensor,
            metadata=converted_metadata,
        )
        return bucket.reconstruct_tensors()

    def _normalize_torch_dtype(self, dtype: Any) -> torch.dtype:
        if isinstance(dtype, torch.dtype):
            return dtype
        if isinstance(dtype, str):
            name = dtype.split(".")[-1]
            normalized = getattr(torch, name, None)
            if isinstance(normalized, torch.dtype):
                return normalized
        raise ValueError(f"Unsupported dtype in flattened_bucket metadata: {dtype!r}")

View on GitHub (pinned to 0132848349)

Solutions

1. Use canonical torch dtype strings: 'torch.float32', 'float16', 'bfloat16'
2. Fix the producer to emit str(tensor.dtype)

Example fix

// before
meta.dtype = "fp32"
// after
meta.dtype = "float32"  # or str(t.dtype)

import torch
name = str(dtype).split(".")[-1]
assert isinstance(getattr(torch, name, None), torch.dtype), f"bad dtype {dtype}"

def dtype_resolvable(dtype: Any) -> bool:
    import torch
    if isinstance(dtype, torch.dtype): return True
    return isinstance(getattr(torch, str(dtype).split(".")[-1], None), torch.dtype)

When it happens

Trigger: Metadata carrying dtype=16 (a numeric id), dtype='fp32', dtype=torch.float32 already fine, but 'FloatTensor' or a numpy dtype string fails; also None dtype.

Common situations: Custom flattener writing numpy dtype ids or shorthand names ('fp16' vs 'float16'); version skew between producer and consumer dtype vocabularies.

Related errors

• flattened_bucket 'flattened_tensor' must be a torch.Tensor.
• flattened_bucket payload must be a dict with 'flattened_tens
• flattened_bucket payload missing 'flattened_tensor' or 'meta
• flattened_bucket 'metadata' must be a list.
• Fused QK-Norm + RoPE kernel only supports float16/bfloat16,

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/b01f0737127554d2. Report an issue: GitHub.