invoke-ai/InvokeAI · error · RuntimeError

Mistral encoder did not return hidden_states. Ensure output_

Error message

Mistral encoder did not return hidden_states. Ensure output_hidden_states=True is supported by this model.

What it means

_encode_prompt throws this RuntimeError when the Mistral encoder's forward pass returns no hidden_states even though it was called with output_hidden_states=True. FLUX.2 [dev] conditioning requires per-layer hidden states for extraction, so a model that cannot supply them is unusable. This typically means the model class does not support the output_hidden_states argument.

Source

Thrown at invokeai/app/invocations/flux2_dev_text_encoder.py:213

            max_length=self.max_seq_len,
        )
        input_ids = inputs["input_ids"].to(device)
        attention_mask = inputs["attention_mask"].to(device)

        # Mistral3ForConditionalGeneration wraps the LM under `.language_model`.
        # For pure text encoding, run that sub-module to skip the (unused) vision
        # tower and to avoid emitting a generation; for plain MistralModel /
        # MistralForCausalLM, run the model directly.
        forward_target = getattr(text_encoder, "language_model", None) or text_encoder

        outputs = forward_target(
            input_ids=input_ids,
            attention_mask=attention_mask,
            output_hidden_states=True,
            use_cache=False,
        )
        if not hasattr(outputs, "hidden_states") or outputs.hidden_states is None:
            raise RuntimeError(
                "Mistral encoder did not return hidden_states. "
                "Ensure output_hidden_states=True is supported by this model."
            )
        num_hidden_states = len(outputs.hidden_states)  # = num_hidden_layers + 1 (embedding output)
        num_layers = num_hidden_states - 1

        # The standalone Mistral encoder loaders only accept 30-layer cow or 40-layer
        # Mistral Small 3 weights, so hidden_states[] should always contain the layers
        # FLUX.2 [dev]'s joint attention was trained to read (10/20/30). A text encoder
        # extracted from a Main_Diffusers_Flux2 pipeline, however, is loaded via generic
        # from_pretrained with no layer-count validation — so a nonstandard pipeline with
        # a <30-layer encoder could reach here. Fail loudly instead of inventing extraction
        # indices that would silently produce off-distribution (degraded) embeddings.
        if num_layers < max(DEV_EXTRACTION_LAYERS):
            raise RuntimeError(
                f"Mistral encoder returned only {num_layers} hidden layer(s), but FLUX.2 [dev] reads "
                f"layers {DEV_EXTRACTION_LAYERS} and requires at least {max(DEV_EXTRACTION_LAYERS)}. "
                "This is not a supported FLUX.2 [dev] text encoder."

View on GitHub (pinned to 0b6a024f2f)

Solutions

Use the standard Mistral Small 3.1 text encoder model/class that supports output_hidden_states.
Upgrade the transformers library to a version where MistralForCausalLM supports output_hidden_states correctly.
Ensure no custom wrapper or patch of the encoder strips the hidden_states output.

Example fix

// before
custom_encoder = load_custom_quantized_encoder()  // ignores output_hidden_states
// after
encoder = MistralForCausalLM.from_pretrained(mistral_3_1_path)

Defensive patterns

Strategy: validation

Validate before calling

outputs = encoder(input_ids=ids, attention_mask=mask, output_hidden_states=True, use_cache=False)
if not hasattr(outputs, "hidden_states") or outputs.hidden_states is None:
    raise RuntimeError("Encoder does not support output_hidden_states")

Type guard

def returns_hidden_states(encoder) -> bool:
    out = encoder(input_ids=torch.tensor([[0]]), output_hidden_states=True, use_cache=False)
    return getattr(out, "hidden_states", None) is not None

Try / catch

try:
    output = text_encoder_invocation.invoke(context)
except RuntimeError as e:
    if "did not return hidden_states" in str(e):
        encoder = load_standard_mistral_encoder()
        output = retry_encoding(encoder)
    else:
        raise

Prevention

Use standard MistralForCausalLM loading rather than custom quantized wrappers.
Smoke-test the encoder once with output_hidden_states=True before wiring it into workflows.
Pin a transformers version known to support Mistral hidden-state outputs.

When it happens

Trigger: The encoder forward call (input_ids, attention_mask, output_hidden_states=True, use_cache=False) completes, but the returned outputs object has no hidden_states attribute or it is None.

Common situations: A custom, quantized, or nonstandard Mistral-compatible model class ignores output_hidden_states; an old transformers version returns an object without the field; or a wrapper model silently drops the option.

Related errors

AI-assisted analysis of invoke-ai/InvokeAI@0b6a024f2f (2026-08-29). Data as JSON: /api/errors/4e2c972c2d982105. Report an issue: GitHub.