sgl-project/sglang · error · ValueError

{name} must stay fp32 after load, got {param.dtype}.

Error message

{name} must stay fp32 after load, got {param.dtype}.

What it means

post_load_weights verifies that numerically sensitive parameters (the AdaLN t-table, adaln_basis, adaln_mean, and similar fp32-mandatory params) remain torch.float32 after weight loading. If weight loading (e.g. a state_dict cast to bf16, or assign=True with a dtype-converted tensor) downcast them, the error names the offending parameter and its dtype.

Source

Thrown at python/sglang/multimodal_gen/runtime/models/dits/minimax_h3.py:2073

            # "ones"); claiming only undeclared params keeps that intact.
            if getattr(param, "missing_param_init", None) is None:
                param.missing_param_init = "error"

    def post_load_weights(self) -> None:
        fp32_param_names = list(_MINIMAX_H3_FP32_PARAM_NAMES_IN_MODEL_ORDER)
        if self.adaln_t_table is not None:
            fp32_param_names = [
                name
                for name in fp32_param_names
                if not name.startswith("time_embedder.")
            ]
            fp32_param_names.append("adaln_t_table")
            if self.adaln_basis is not None:
                fp32_param_names.extend(("adaln_basis", "adaln_mean"))
        for name in fp32_param_names:
            param = self.get_parameter(name)
            if param.dtype != _FP32_DTYPE:
                raise ValueError(
                    f"{name} must stay fp32 after load, got {param.dtype}."
                )
        if self.adaln_t_table is not None:
            for name, param in self.named_parameters():
                if ".adaln_proj.linear." in name and param.dtype != _FP32_DTYPE:
                    raise ValueError(
                        f"{name} must stay fp32 with curve AdaLN, got {param.dtype}."
                    )
        # assign=True loading may re-register this persistent buffer as a parameter
        rope_inv_freq = self.rope.inv_freq
        if rope_inv_freq.dtype != _FP32_DTYPE:
            raise ValueError(
                f"rope.inv_freq must stay fp32 after load, got {rope_inv_freq.dtype}."
            )
        if self.adaln_cache is not None:
            self.adaln_cache.load(self.video_patch_proj.weight.device)

    def _time_embedding(self, timesteps: torch.Tensor) -> torch.Tensor:

View on GitHub (pinned to 0132848349)

Solutions

  1. Keep the fp32-mandatory tensors fp32 in the checkpoint (re-save without global dtype casting)
  2. Load with assign but cast only non-fp32-mandatory params, letting post_load_weights re-assert dtypes
  3. If converting, whitelist: adaln_t_table, adaln_basis, adaln_mean (and other listed names) from any dtype downcast

Example fix

# before
state = {k: v.to(torch.bfloat16) for k, v in state.items()}
model.load_state_dict(state, assign=True)
# after
FP32_KEEP = {"adaln_t_table", "adaln_basis", "adaln_mean"}
state = {k: (v if k in FP32_KEEP else v.to(torch.bfloat16)) for k, v in state.items()}
Defensive patterns

Strategy: type-guard

Validate before calling

FP32_KEEP = {"adaln_t_table", "adaln_basis", "adaln_mean"}
for name, p in model.named_parameters():
    if name in FP32_KEEP:
        assert p.dtype == torch.float32, (name, p.dtype)

Type guard

def fp32_params_ok(model) -> bool:
    keep = {"adaln_t_table", "adaln_basis", "adaln_mean"}
    return all(p.dtype == torch.float32 for n, p in model.named_parameters() if n in keep)

Try / catch

try:
    model.post_load_weights()
except ValueError as e:
    for n, p in model.named_parameters():
        if n in FP32_KEEP:
            p.data = p.data.float()
    model.post_load_weights()

Prevention

When it happens

Trigger: Loading a checkpoint whose fp32 params (adaln_t_table etc.) were saved/loaded as bf16/fp16; using a loader that casts the whole state_dict to the compute dtype before assign.

Common situations: Pre-casting checkpoints to bf16 to shrink files; custom load hooks applying model.to(torch.bfloat16) before post_load_weights; conversion scripts that rewrite tensors with torch.Tensor.to(dtype).

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/63ab3eb6fe3511de. Report an issue: GitHub.