invoke-ai/InvokeAI · error · TypeError

Expected torch.Tensor for attention_mask, got {type(attentio

Error message

Expected torch.Tensor for attention_mask, got {type(attention_mask).__name__}. Tokenizer returned unexpected type.

What it means

Immediately after the input_ids check, _encode_prompt validates that text_inputs.attention_mask is a torch.Tensor. A non-tensor attention_mask raises a TypeError stating the tokenizer returned an unexpected type. The attention mask is required for the padded Qwen3 forward pass, so its type must be guaranteed.

Source

Thrown at invokeai/app/invocations/z_image_text_encoder.py:149

            # Tokenize the formatted prompt
            text_inputs = tokenizer(
                prompt_formatted,
                padding="max_length",
                max_length=max_seq_len,
                truncation=True,
                return_attention_mask=True,
                return_tensors="pt",
            )

            text_input_ids = text_inputs.input_ids
            attention_mask = text_inputs.attention_mask
            if not isinstance(text_input_ids, torch.Tensor):
                raise TypeError(
                    f"Expected torch.Tensor for input_ids, got {type(text_input_ids).__name__}. "
                    "Tokenizer returned unexpected type."
                )
            if not isinstance(attention_mask, torch.Tensor):
                raise TypeError(
                    f"Expected torch.Tensor for attention_mask, got {type(attention_mask).__name__}. "
                    "Tokenizer returned unexpected type."
                )

            # Check for truncation
            untruncated_ids = tokenizer(prompt_formatted, padding="longest", return_tensors="pt").input_ids
            if untruncated_ids.shape[-1] >= text_input_ids.shape[-1] and not torch.equal(
                text_input_ids, untruncated_ids
            ):
                removed_text = tokenizer.batch_decode(untruncated_ids[:, max_seq_len - 1 : -1])
                context.logger.warning(
                    f"The following part of your input was truncated because `max_sequence_length` is set to "
                    f"{max_seq_len} tokens: {removed_text}"
                )

            # Get hidden states from the text encoder
            # Use the second-to-last hidden state like diffusers does
            prompt_mask = attention_mask.to(device).bool()

View on GitHub (pinned to 0b6a024f2f)

Solutions

Use the stock transformers Qwen3 tokenizer so attention_mask is tensorized with return_tensors="pt".
Upgrade transformers if return_tensors is being ignored.
Validate the tokenizer output before passing to the invocation (isinstance check on attention_mask).
Convert manually if required: attention_mask = torch.tensor(raw_mask).

Example fix

// before
text_inputs = tok(prompt)  # returns lists
// after
text_inputs = tok(prompt, padding="longest", return_tensors="pt")  # tensors

Defensive patterns

Strategy: type-guard

Validate before calling

inputs = tok(prompt, padding="longest", return_tensors="pt")
if not isinstance(inputs.attention_mask, torch.Tensor):
    fail_fast(inputs.attention_mask)

Type guard

def is_tensor(x) -> bool:
    import torch
    return isinstance(x, torch.Tensor)

Try / catch

try:
    encode(context)
except TypeError as e:
    if "Expected torch.Tensor for attention_mask" in str(e):
        swap_to_stock_tokenizer()
    else:
        raise

Prevention

Use stock transformers tokenizers so attention_mask is tensorized automatically.
Never construct BatchEncoding manually without tensor conversion.
Pin a transformers version known to work with your tokenizer files.

When it happens

Trigger: Z-Image text encoding where tokenizer(..., padding="longest", return_tensors="pt").attention_mask is not a torch.Tensor, e.g. returned as a list/ndarray by a custom tokenizer.

Common situations: Custom tokenizer subclass bypassing tensor conversion; transformers version mismatch; manually constructed BatchEncoding lacking tensorized fields.

Related errors

AI-assisted analysis of invoke-ai/InvokeAI@0b6a024f2f (2026-08-29). Data as JSON: /api/errors/ecad3745ef5eba35. Report an issue: GitHub.