sgl-project/sglang · error · ValueError

flattened_bucket 'flattened_tensor' must be a torch.Tensor.

Error message

flattened_bucket 'flattened_tensor' must be a torch.Tensor.

What it means

Raised by WeightsUpdater._reconstruct_from_flattened_bucket when the 'flattened_tensor' field of a flattened_bucket payload is not a torch.Tensor. The reconstruction logic slices this tensor using metadata offsets, so it must be a real tensor.

Source

Thrown at python/sglang/multimodal_gen/runtime/post_training/weights_updater.py:790

            flattened_tensor = module_payload.get("flattened_tensor")
            metadata = module_payload.get("metadata")
            if flattened_tensor is None or metadata is None:
                raise ValueError(
                    "flattened_bucket payload missing 'flattened_tensor' or 'metadata'."
                )
            return self._reconstruct_from_flattened_bucket(flattened_tensor, metadata)

        if isinstance(module_payload, (list, tuple)):
            return iter(module_payload)

        raise ValueError(
            f"Unsupported module payload type for load_format={load_format}: "
            f"{type(module_payload).__name__}"
        )

    def _reconstruct_from_flattened_bucket(self, flattened_tensor: Any, metadata: Any):
        if not isinstance(flattened_tensor, torch.Tensor):
            raise ValueError(
                "flattened_bucket 'flattened_tensor' must be a torch.Tensor."
            )
        if not isinstance(metadata, list):
            raise ValueError("flattened_bucket 'metadata' must be a list.")

        converted_metadata: list[FlattenedTensorMetadata] = []
        for meta in metadata:
            converted_metadata.append(
                FlattenedTensorMetadata(
                    name=meta.name,
                    shape=torch.Size(meta.shape),
                    dtype=self._normalize_torch_dtype(meta.dtype),
                    start_idx=int(meta.start_idx),
                    end_idx=int(meta.end_idx),
                    numel=int(meta.numel),
                )
            )

View on GitHub (pinned to 0132848349)

Solutions

1. Convert before sending: torch.tensor(payload['flattened_tensor']) or keep the tensor via a binary-safe channel
2. On the receiver, wrap deserialized arrays back into torch.Tensor before calling the updater

Example fix

// before
bucket["flattened_tensor"] = arr.numpy()  # or list
// after
bucket["flattened_tensor"] = torch.from_numpy(arr)

import torch
if not isinstance(bucket["flattened_tensor"], torch.Tensor):
    bucket["flattened_tensor"] = torch.as_tensor(bucket["flattened_tensor"])

def tensor_is_ok(t: Any) -> bool:
    import torch; return isinstance(t, torch.Tensor)

When it happens

Trigger: load_format='flattened_bucket' with 'flattened_tensor' holding a numpy array, a list of floats, or a string (e.g. after JSON serialization of the bucket).

Common situations: Sending the bucket over a wire protocol (ZMQ/HTTP/JSON) that serializes tensors to lists/bytes without deserializing them back.

Related errors

• Unsupported dtype in flattened_bucket metadata: {dtype!r}
• flattened_bucket payload must be a dict with 'flattened_tens
• flattened_bucket payload missing 'flattened_tensor' or 'meta
• flattened_bucket 'metadata' must be a list.
• Fused QK-Norm + RoPE kernel only supports float16/bfloat16,

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/e93f379ccd03ca40. Report an issue: GitHub.