sgl-project/sglang · error · ValueError

rope.inv_freq must stay fp32 after load, got {rope_inv_freq.

Error message

rope.inv_freq must stay fp32 after load, got {rope_inv_freq.dtype}.

What it means

The rotary-embedding inverse-frequency buffer (rope.inv_freq) must remain fp32; with assign=True loading PyTorch may re-register this persistent buffer from the incoming state_dict, and if that tensor is bf16/fp16 the post-load check fails. fp32 inv_freq is required for positional-frequency numerical accuracy.

Source

Thrown at python/sglang/multimodal_gen/runtime/models/dits/minimax_h3.py:2085

            fp32_param_names.append("adaln_t_table")
            if self.adaln_basis is not None:
                fp32_param_names.extend(("adaln_basis", "adaln_mean"))
        for name in fp32_param_names:
            param = self.get_parameter(name)
            if param.dtype != _FP32_DTYPE:
                raise ValueError(
                    f"{name} must stay fp32 after load, got {param.dtype}."
                )
        if self.adaln_t_table is not None:
            for name, param in self.named_parameters():
                if ".adaln_proj.linear." in name and param.dtype != _FP32_DTYPE:
                    raise ValueError(
                        f"{name} must stay fp32 with curve AdaLN, got {param.dtype}."
                    )
        # assign=True loading may re-register this persistent buffer as a parameter
        rope_inv_freq = self.rope.inv_freq
        if rope_inv_freq.dtype != _FP32_DTYPE:
            raise ValueError(
                f"rope.inv_freq must stay fp32 after load, got {rope_inv_freq.dtype}."
            )
        if self.adaln_cache is not None:
            self.adaln_cache.load(self.video_patch_proj.weight.device)

    def _time_embedding(self, timesteps: torch.Tensor) -> torch.Tensor:
        if self.adaln_t_table is None:
            assert self.time_embedder is not None
            return self.time_embedder(timesteps)

        grid = self.adaln_t_table.shape[0]
        position = timesteps.to(_FP32_DTYPE).clamp(0, 1) * (grid - 1)
        lower = position.floor().clamp(max=grid - 2).to(torch.long)
        fraction = (position - lower).unsqueeze(-1)
        lower_value = self.adaln_t_table.index_select(0, lower)
        upper_value = self.adaln_t_table.index_select(0, lower + 1)
        return torch.lerp(lower_value, upper_value, fraction)

View on GitHub (pinned to 0132848349)

Solutions

Re-save the checkpoint with rope.inv_freq in fp32, or drop inv_freq from the state_dict so the module's own fp32 buffer survives
After assign=True loading, explicitly restore: model.rope.inv_freq = model.rope.inv_freq.float() before calling post_load_weights
Avoid global dtype casts when materializing state_dict tensors

Example fix

# before
sd = torch.load(p, map_location="cpu", dtype=torch.bfloat16)  # casts inv_freq
# after
sd = torch.load(p, map_location="cpu")
sd.pop("rope.inv_freq", None)  # let module keep its fp32 buffer

Defensive patterns

Strategy: type-guard

Validate before calling

sd.pop("rope.inv_freq", None)
assert model.rope.inv_freq.dtype == torch.float32

Type guard

def inv_freq_fp32(model) -> bool:
    return model.rope.inv_freq.dtype == torch.float32

Try / catch

try:
    model.post_load_weights()
except ValueError:
    model.rope.inv_freq = model.rope.inv_freq.float()
    model.post_load_weights()

Prevention

Exclude rope.inv_freq from dtype casts and checkpoints
After assign=True loads, re-assert buffer dtypes

When it happens

Trigger: Loading a state_dict with assign=True where 'rope.inv_freq' was saved in a reduced dtype, or where load re-registers it as a parameter with the wrong dtype.

Common situations: Half-precision checkpoint exports that included inv_freq; custom loaders that materialize all tensors in the compute dtype; newer PyTorch behavior changes around persistent buffers with assign=True.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/f8dbd58d6693442b. Report an issue: GitHub.