{"record":{"id":"8d2c3b482d0db10a","repo":"sgl-project/sglang","slug":"name-must-stay-fp32-with-curve-adaln-got-param","errorCode":null,"errorMessage":"{name} must stay fp32 with curve AdaLN, got {param.dtype}.","messagePattern":"(.+?) must stay fp32 with curve AdaLN, got (.+?)\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/models/dits/minimax_h3.py","lineNumber":2079,"sourceCode":"        if self.adaln_t_table is not None:\n            fp32_param_names = [\n                name\n                for name in fp32_param_names\n                if not name.startswith(\"time_embedder.\")\n            ]\n            fp32_param_names.append(\"adaln_t_table\")\n            if self.adaln_basis is not None:\n                fp32_param_names.extend((\"adaln_basis\", \"adaln_mean\"))\n        for name in fp32_param_names:\n            param = self.get_parameter(name)\n            if param.dtype != _FP32_DTYPE:\n                raise ValueError(\n                    f\"{name} must stay fp32 after load, got {param.dtype}.\"\n                )\n        if self.adaln_t_table is not None:\n            for name, param in self.named_parameters():\n                if \".adaln_proj.linear.\" in name and param.dtype != _FP32_DTYPE:\n                    raise ValueError(\n                        f\"{name} must stay fp32 with curve AdaLN, got {param.dtype}.\"\n                    )\n        # assign=True loading may re-register this persistent buffer as a parameter\n        rope_inv_freq = self.rope.inv_freq\n        if rope_inv_freq.dtype != _FP32_DTYPE:\n            raise ValueError(\n                f\"rope.inv_freq must stay fp32 after load, got {rope_inv_freq.dtype}.\"\n            )\n        if self.adaln_cache is not None:\n            self.adaln_cache.load(self.video_patch_proj.weight.device)\n\n    def _time_embedding(self, timesteps: torch.Tensor) -> torch.Tensor:\n        if self.adaln_t_table is None:\n            assert self.time_embedder is not None\n            return self.time_embedder(timesteps)\n\n        grid = self.adaln_t_table.shape[0]\n        position = timesteps.to(_FP32_DTYPE).clamp(0, 1) * (grid - 1)","sourceCodeStart":2061,"sourceCodeEnd":2097,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/models/dits/minimax_h3.py#L2061-L2097","documentation":"When curve AdaLN is active (adaln_t_table present), every parameter under a '.adaln_proj.linear.' submodule must stay fp32 because the curve interpolation math is dtype-sensitive. post_load_weights scans named_parameters and raises on any such parameter that was downcast during loading.","triggerScenarios":"Loading a curve-AdaLN checkpoint where adaln_proj.linear.weight/bias arrived as bf16/fp16 — usually from a global dtype cast in the loader or a mixed-precision checkpoint.","commonSituations":"model.to(torch.bfloat16) before load; converters that cast all linear weights; fp16-serialized checkpoints.","solutions":["Exclude adaln_proj.linear.* tensors from any dtype casting in your conversion/loading pipeline","Re-save the checkpoint keeping adaln_proj weights in fp32","Load in the native checkpoint dtype and cast only compute weights afterwards"],"exampleFix":"# before\nmodel.to(torch.bfloat16); model.load_state_dict(sd, assign=True)\n# after\nmodel.load_state_dict(sd, assign=True)\nmodel._keep_adaln_fp32()  # or cast everything except adaln_proj/adaln tables","handlingStrategy":"type-guard","validationCode":"for name, p in model.named_parameters():\n    if \".adaln_proj.linear.\" in name:\n        assert p.dtype == torch.float32, (name, p.dtype)","typeGuard":"def adaln_proj_fp32(model) -> bool:\n    return all(p.dtype == torch.float32\n               for n, p in model.named_parameters()\n               if \".adaln_proj.linear.\" in n)","tryCatchPattern":"try:\n    model.post_load_weights()\nexcept ValueError:\n    with torch.no_grad():\n        for n, p in model.named_parameters():\n            if \".adaln_proj.linear.\" in n:\n                p.data = p.data.float()\n    model.post_load_weights()","preventionTips":["Cast compute weights only after loading","Keep curve-AdaLN checkpoints' adaln_proj tensors fp32"],"tags":["dtype","fp32","adaln","weight-loading"],"backgroundTag":"weight-dtype-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}