{"record":{"id":"ed6f0cc03ef7cb8b","repo":"sgl-project/sglang","slug":"post-load-processing-produced-a-meta-tensor","errorCode":null,"errorMessage":"Post-load processing produced a meta tensor","messagePattern":"Post-load processing produced a meta tensor","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"critical","filePath":"python/sglang/srt/model_loader/post_load.py","lineNumber":89,"sourceCode":"            dtype=data.dtype,\n            layout=data.layout,\n            device=device,\n            pin_memory=pin_memory,\n        )\n        result.copy_(data)\n        return result\n    return data.to(device)\n\n\ndef _restore_tensor(\n    tensor: torch.Tensor,\n    destination: torch.device,\n    original_state: _TensorState | None,\n    *,\n    pin_memory: bool,\n) -> None:\n    if tensor.is_meta:\n        raise RuntimeError(\"Post-load processing produced a meta tensor\")\n\n    if (\n        original_state is not None\n        and tensor is original_state.tensor\n        and original_state.staged_data is not None\n        and _same_staged_data(tensor.data, original_state.staged_data)\n    ):\n        original_state.original_data.copy_(tensor.data)\n        tensor.data = original_state.original_data\n        return\n\n    tensor.data = _copy_data_to_device(\n        tensor.data,\n        destination,\n        pin_memory=pin_memory,\n    )\n\n","sourceCodeStart":71,"sourceCodeEnd":107,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/model_loader/post_load.py#L71-L107","documentation":"During post-load processing (post_load.py), _restore_tensor checks every tensor before restoring it to its destination device; if a tensor is still on the meta device it means the load/post-processing pipeline never materialized real data for it — typically a weight was never loaded or a transform returned a meta tensor. The RuntimeError is a hard integrity check, not transient.","triggerScenarios":"stage_module_for_post_load staged a module where some parameter/buffer was left uninitialized (meta) — e.g. a weight file was skipped, a quantization/precision transform created a new meta tensor, or a loader path forgot to empty_like().copy_ real data before restore.","commonSituations":"Custom loader or custom model code that creates parameters with torch.device('meta') and forgets to materialize; partial/broken checkpoint shards; bugs in post-load transforms (per-op quant, dtype cast) after an sglang upgrade; missing keys in the checkpoint not caught earlier.","solutions":["Identify the meta tensor: before restore, scan named_parameters for p.is_meta and log the name to find which module/weight was skipped","Ensure the checkpoint actually contains that weight (check safetensors index / missing keys in load logs)","If writing custom post-load transforms, materialize outputs with torch.empty_like(t, device='cpu') and copy real data instead of returning meta tensors","Update sglang — this check guards known loader regressions; if it fires on stock models, report with the model + config"],"exampleFix":"# before (custom transform returns meta tensor)\ndef transform(t):\n    return torch.empty(t.shape, dtype=t.dtype, device=\"meta\")  # never materialized\n\n# after\ndef transform(t):\n    out = torch.empty_like(t, device=\"cpu\")\n    out.copy_(t)\n    return out","handlingStrategy":"validation","validationCode":"meta = [(n, p) for n, p in model.named_parameters() if p.is_meta]\nmeta += [(n, b) for n, b in model.named_buffers() if b.is_meta]\nif meta:\n    raise RuntimeError(f\"Unmaterialized meta tensors before restore: {[n for n, _ in meta]}\")","typeGuard":"def module_fully_materialized(module: torch.nn.Module) -> bool:\n    return not any(t.is_meta for t in list(module.parameters()) + list(module.buffers()))","tryCatchPattern":"try:\n    stage_module_for_post_load(model, ...)\nexcept RuntimeError as e:\n    if \"meta tensor\" in str(e):\n        # locate & explicitly materialize the skipped weight, then retry once\n        raise","preventionTips":["In custom loaders/transforms, always allocate real tensors (empty_like + copy_) never meta","Check load logs for missing/unexpected keys before post-load runs","Diff checkpoint shard list vs model state_dict keys in CI for supported checkpoints"],"tags":["meta-tensor","post-load","uninitialized-weights","model-loader","sglang"],"backgroundTag":"uninitialized-model-weights","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}