{"record":{"id":"3977e9a48ca539c5","repo":"sgl-project/sglang","slug":"sana-wm-refiner-text-encoder-must-return-per-layer","errorCode":null,"errorMessage":"SANA-WM refiner text encoder must return per-layer hidden_states.","messagePattern":"SANA-WM refiner text encoder must return per-layer hidden_states\\.","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/pipelines_core/stages/model_specific_stages/sana_wm/refiner.py","lineNumber":520,"sourceCode":"        )\n        input_ids = text_inputs.input_ids.to(device)\n        attention_mask = text_inputs.attention_mask.to(device)\n\n        # Diffusers-backed official path loads HF Gemma3ForConditionalGeneration.\n        # NVlabs encodes through `.model`; the fallback SGLang-native encoder is\n        # still callable directly, so keep both surfaces.\n        with self.use_declared_component(\n            component_name=\"text_encoder_2\", module=self.text_encoder\n        ):\n            text_backbone = getattr(self.text_encoder, \"model\", self.text_encoder)\n            outputs = text_backbone(\n                input_ids=input_ids,\n                attention_mask=attention_mask,\n                output_hidden_states=True,\n            )\n        per_layer_hidden = getattr(outputs, \"hidden_states\", None)\n        if per_layer_hidden is None:\n            raise RuntimeError(\n                \"SANA-WM refiner text encoder must return per-layer hidden_states.\"\n            )\n        stacked = torch.stack(per_layer_hidden, dim=-1)  # (B, L, D, n_layers)\n        seq_lengths = attention_mask.sum(dim=-1)\n        log_sana_wm_tensor_stats(\"refiner.text_hidden_states_stacked\", stacked)\n        prompt_embeds = _pack_text_embeds(\n            stacked,\n            seq_lengths,\n            padding_side=tokenizer.padding_side,\n        ).to(dtype=self.dtype)\n        log_sana_wm_tensor_stats(\"refiner.prompt_embeds_packed\", prompt_embeds)\n\n        with self.use_declared_component(\n            component_name=\"connectors\", module=self.connectors\n        ):\n            video_text_embedding, _, video_attention_mask = self.connectors(\n                prompt_embeds, attention_mask\n            )","sourceCodeStart":502,"sourceCodeEnd":538,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/pipelines_core/stages/model_specific_stages/sana_wm/refiner.py#L502-L538","documentation":"_encode_prompt calls the text encoder with output_hidden_states=True and then reads outputs.hidden_states. If the encoder's returned object lacks per-layer hidden_states (attribute None or absent), the refiner cannot build the stacked (B, L, D, n_layers) embedding tensor the SANA-WM refiner consumes, so it raises this RuntimeError.","triggerScenarios":"Configuring the refiner with a text encoder whose forward signature/return type does not expose hidden_states (custom encoder wrapper, wrong model class, diffusers version returning a different output dataclass, or a wrapper that drops the field).","commonSituations":"Swapping in a different text encoder via component overrides; upgrading/downgrading transformers where model output objects changed; a custom wrapper that returns only last_hidden_state.","solutions":["Use the supported text encoder class for the SANA-WM pipeline","If wrapping the encoder, return an object with .hidden_states populated (pass output_hidden_states=True through and forward the tuple)","Check the wrapper returns the encoder's native output object rather than a reduced one"],"exampleFix":"# before\nclass Wrapper(nn.Module):\n    def forward(self, **kw):\n        out = self.enc(**kw)\n        return out.last_hidden_state  # hidden_states lost\n# after\nclass Wrapper(nn.Module):\n    def forward(self, **kw):\n        kw[\"output_hidden_states\"] = True\n        return self.enc(**kw)  # exposes .hidden_states","handlingStrategy":"type-guard","validationCode":"out = encoder(input_ids=ids, attention_mask=mask, output_hidden_states=True)\nassert getattr(out, \"hidden_states\", None) is not None, \"encoder must expose per-layer hidden_states\"","typeGuard":"def exposes_hidden_states(enc) -> bool:\n    import inspect\n    out = enc(input_ids=torch.zeros(1, 4, dtype=torch.long), attention_mask=torch.ones(1, 4, dtype=torch.long), output_hidden_states=True)\n    return getattr(out, \"hidden_states\", None) is not None","tryCatchPattern":"try:\n    return stage._encode_prompt(ids, mask)\nexcept RuntimeError as e:\n    if \"hidden_states\" in str(e):\n        raise TypeError(f\"encoder {type(encoder).__name__} incompatible: {e}\")\n    raise","preventionTips":["Smoke-test any encoder swap with output_hidden_states=True once at startup","Prefer returning the encoder's native output object from wrappers","Pin the transformers/diffusers versions the pipeline was validated with"],"tags":["sana-wm","text-encoder","hidden-states","runtimeerror","model-output"],"backgroundTag":"missing-model-output-field","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}