sgl-project/sglang · error · ValueError

Unknown {old_param_type=} {old_param=}

Error message

Unknown {old_param_type=} {old_param=}

What it means

_move_param_to_meta only knows how to move torch.nn.Parameter and torch.Tensor instances onto the meta device; any other object found in a module attribute fails with this ValueError. It is an internal path used by the offloader during init/post_init.

Source

Thrown at python/sglang/srt/utils/offloader.py:472

        # manually checked how `w13_weight` and `w2_weight` are constructed
        new_param = ModelWeightParameter(
            data=new_data,
            **{
                k: getattr(old_param, k)
                for k in ["input_dim", "output_dim", "weight_loader"]
            },
        )
    elif old_param_type == torch.nn.Parameter:
        new_param = torch.nn.Parameter(
            data=new_data,
            requires_grad=False,
        )
        if hasattr(old_param, "weight_loader"):
            new_param.weight_loader = old_param.weight_loader
        else:
            new_param.weight_loader = lambda *args, **kwargs: None
    else:
        raise ValueError(f"Unknown {old_param_type=} {old_param=}")

    setattr(module, param_name, new_param)


def _empty_strided_like(x: torch.Tensor, device, pin_memory=False):
    return torch.empty_strided(
        size=x.size(),
        stride=x.stride(),
        dtype=x.dtype,
        layout=x.layout,
        device=device,
        pin_memory=pin_memory,
    )


# ----------------------------------------- ShardedGpu ------------------------------------------------------

View on GitHub (pinned to 0132848349)

Solutions

1. Upgrade sglang — quantized/DTensor param types are handled in newer offloader code
2. Ensure offloaded modules hold only standard Parameter/Tensor attributes
3. If writing custom layers, register exotic buffers via register_buffer with plain tensors

Example fix

# before
class MyLayer(nn.Module):
    self.weight = QuantParam(...)  # not Parameter/Tensor
# after
class MyLayer(nn.Module):
    self.weight = nn.Parameter(...)
    self.qstate = ...  # keep non-tensors out of param slots

import torch
assert all(isinstance(p, (torch.nn.Parameter, torch.Tensor)) for p in module.parameters(recurse=False))

import torch
def is_offloadable_param(obj) -> bool:
    return isinstance(obj, (torch.nn.Parameter, torch.Tensor))

try:
    offloader._move_param_to_meta(module, name)
except ValueError as e:
    if 'Unknown old_param_type' in str(e):
        skip.add(name)  # skip unsupported param types

When it happens

Trigger: A module attribute that looks like a parameter slot holds a non-tensor object (e.g. a custom DTensor/QuantizedParameter subclass or a plain object) when the offloader walks module parameters.

Common situations: Custom quantization or DTensor-wrapped parameters whose type is neither Parameter nor Tensor; version drift introducing new param container types in external libs.

Related errors

• NCCL only supports CUDA, ROCm and MUSA backends.
• Unknown serve backend {name!r}. Available values: {available
• Multiple distributions register serve backend {name!r}: {pro
• Failed to load serve backend {name!r} from {self._entry_poin
• Serve backend {name!r} factory returned {type(backend).__nam

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/89fbe78abd207d00. Report an issue: GitHub.