invoke-ai/InvokeAI · error · RuntimeError

Expected at least 2 hidden states from text encoder, got {le

Error message

Expected at least 2 hidden states from text encoder, got {len(outputs.hidden_states)}. This may indicate an incompatible model or configuration.

What it means

After confirming hidden_states exists, _encode_prompt requires at least 2 entries because it takes hidden_states[-2] (the second-to-last layer) as prompt_embeds. Fewer than 2 indicates the encoder ran with an incompatible configuration or model, and a RuntimeError reports the actual count.

Source

Thrown at invokeai/app/invocations/z_image_text_encoder.py:181

                )

            # Get hidden states from the text encoder
            # Use the second-to-last hidden state like diffusers does
            prompt_mask = attention_mask.to(device).bool()
            outputs = text_encoder(
                text_input_ids.to(device),
                attention_mask=prompt_mask,
                output_hidden_states=True,
            )

            # Validate hidden_states output
            if not hasattr(outputs, "hidden_states") or outputs.hidden_states is None:
                raise RuntimeError(
                    "Text encoder did not return hidden_states. "
                    "Ensure output_hidden_states=True is supported by this model."
                )
            if len(outputs.hidden_states) < 2:
                raise RuntimeError(
                    f"Expected at least 2 hidden states from text encoder, got {len(outputs.hidden_states)}. "
                    "This may indicate an incompatible model or configuration."
                )
            prompt_embeds = outputs.hidden_states[-2]

            # Z-Image expects a 2D tensor [seq_len, hidden_dim] with only valid tokens
            # Based on diffusers ZImagePipeline implementation:
            # embeddings_list.append(prompt_embeds[i][prompt_masks[i]])
            # Since batch_size=1, we take the first item and filter by mask
            prompt_embeds = prompt_embeds[0][prompt_mask[0]]

        if not isinstance(prompt_embeds, torch.Tensor):
            raise TypeError(
                f"Expected torch.Tensor for prompt embeddings, got {type(prompt_embeds).__name__}. "
                "Text encoder returned unexpected type."
            )
        return prompt_embeds

View on GitHub (pinned to 0b6a024f2f)

Solutions

1. Use the full Qwen3 encoder checkpoint intended for Z-Image (with multiple hidden layers).
2. Verify the model config's num_hidden_layers > 0 and re-download config.json if suspect.
3. Re-import the Z-Image model so the correct text encoder submodel is bound.
4. Update transformers/InvokeAI so all layer hidden states are collected.

outputs = encoder(**inputs, output_hidden_states=True)
if len(outputs.hidden_states) < 2:
    fail_fast(f"need >=2 hidden states, got {len(outputs.hidden_states)}")

try:
    encode(context)
except RuntimeError as e:
    if "Expected at least 2 hidden states" in str(e):
        reload_full_qwen3_checkpoint()
    else:
        raise

When it happens

Trigger: Z-Image text encoding where outputs.hidden_states has length < 2 (e.g. length 1: only embedding output, or 0) despite output_hidden_states=True.

Common situations: A truncated/tiny Qwen3 model (0 hidden layers) or misconfigured num_hidden_layers=0; incompatible checkpoint converted incorrectly; model wrapper stripping hidden layers; wrong submodel selected.

Related errors

• Text encoder did not return hidden_states. Ensure output_hid
• LoRA "{lora_key}" already applied to Qwen3 encoder.
• Expected PreTrainedModel for text encoder, got {type(text_en
• Expected PreTrainedTokenizerBase for tokenizer, got {type(to
• Text encoder did not return hidden_states.

AI-assisted analysis of invoke-ai/InvokeAI@0b6a024f2f (2026-08-29). Data as JSON: /api/errors/39fc11581f69e5a5. Report an issue: GitHub.