sgl-project/sglang · critical · RuntimeError

Post-load processing produced a meta tensor

Error message

Post-load processing produced a meta tensor

What it means

During post-load processing (post_load.py), _restore_tensor checks every tensor before restoring it to its destination device; if a tensor is still on the meta device it means the load/post-processing pipeline never materialized real data for it — typically a weight was never loaded or a transform returned a meta tensor. The RuntimeError is a hard integrity check, not transient.

Source

Thrown at python/sglang/srt/model_loader/post_load.py:89

            dtype=data.dtype,
            layout=data.layout,
            device=device,
            pin_memory=pin_memory,
        )
        result.copy_(data)
        return result
    return data.to(device)


def _restore_tensor(
    tensor: torch.Tensor,
    destination: torch.device,
    original_state: _TensorState | None,
    *,
    pin_memory: bool,
) -> None:
    if tensor.is_meta:
        raise RuntimeError("Post-load processing produced a meta tensor")

    if (
        original_state is not None
        and tensor is original_state.tensor
        and original_state.staged_data is not None
        and _same_staged_data(tensor.data, original_state.staged_data)
    ):
        original_state.original_data.copy_(tensor.data)
        tensor.data = original_state.original_data
        return

    tensor.data = _copy_data_to_device(
        tensor.data,
        destination,
        pin_memory=pin_memory,
    )

View on GitHub (pinned to 0132848349)

Solutions

1. Identify the meta tensor: before restore, scan named_parameters for p.is_meta and log the name to find which module/weight was skipped
2. Ensure the checkpoint actually contains that weight (check safetensors index / missing keys in load logs)
3. If writing custom post-load transforms, materialize outputs with torch.empty_like(t, device='cpu') and copy real data instead of returning meta tensors
4. Update sglang — this check guards known loader regressions; if it fires on stock models, report with the model + config

Example fix

# before (custom transform returns meta tensor)
def transform(t):
    return torch.empty(t.shape, dtype=t.dtype, device="meta")  # never materialized

# after
def transform(t):
    out = torch.empty_like(t, device="cpu")
    out.copy_(t)
    return out

meta = [(n, p) for n, p in model.named_parameters() if p.is_meta]
meta += [(n, b) for n, b in model.named_buffers() if b.is_meta]
if meta:
    raise RuntimeError(f"Unmaterialized meta tensors before restore: {[n for n, _ in meta]}")

def module_fully_materialized(module: torch.nn.Module) -> bool:
    return not any(t.is_meta for t in list(module.parameters()) + list(module.buffers()))

try:
    stage_module_for_post_load(model, ...)
except RuntimeError as e:
    if "meta tensor" in str(e):
        # locate & explicitly materialize the skipped weight, then retry once
        raise

When it happens

Trigger: stage_module_for_post_load staged a module where some parameter/buffer was left uninitialized (meta) — e.g. a weight file was skipped, a quantization/precision transform created a new meta tensor, or a loader path forgot to empty_like().copy_ real data before restore.

Common situations: Custom loader or custom model code that creates parameters with torch.device('meta') and forgets to materialize; partial/broken checkpoint shards; bugs in post-load transforms (per-op quant, dtype cast) after an sglang upgrade; missing keys in the checkpoint not caught earlier.

Related errors

• Unknown serve backend {name!r}. Available values: {available
• Multiple distributions register serve backend {name!r}: {pro
• Failed to load serve backend {name!r} from {self._entry_poin
• Serve backend {name!r} factory returned {type(backend).__nam
• Serve backend {name!r} uses API version {backend.api_version

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/ed6f0cc03ef7cb8b. Report an issue: GitHub.