{"record":{"id":"ecad3745ef5eba35","repo":"invoke-ai/InvokeAI","slug":"expected-torch-tensor-for-attention-mask-got-typ","errorCode":null,"errorMessage":"Expected torch.Tensor for attention_mask, got {type(attention_mask).__name__}. Tokenizer returned unexpected type.","messagePattern":"Expected torch\\.Tensor for attention_mask, got (.+?)\\. Tokenizer returned unexpected type\\.","errorType":"exception","errorClass":"TypeError","httpStatus":null,"severity":"error","filePath":"invokeai/app/invocations/z_image_text_encoder.py","lineNumber":149,"sourceCode":"            # Tokenize the formatted prompt\n            text_inputs = tokenizer(\n                prompt_formatted,\n                padding=\"max_length\",\n                max_length=max_seq_len,\n                truncation=True,\n                return_attention_mask=True,\n                return_tensors=\"pt\",\n            )\n\n            text_input_ids = text_inputs.input_ids\n            attention_mask = text_inputs.attention_mask\n            if not isinstance(text_input_ids, torch.Tensor):\n                raise TypeError(\n                    f\"Expected torch.Tensor for input_ids, got {type(text_input_ids).__name__}. \"\n                    \"Tokenizer returned unexpected type.\"\n                )\n            if not isinstance(attention_mask, torch.Tensor):\n                raise TypeError(\n                    f\"Expected torch.Tensor for attention_mask, got {type(attention_mask).__name__}. \"\n                    \"Tokenizer returned unexpected type.\"\n                )\n\n            # Check for truncation\n            untruncated_ids = tokenizer(prompt_formatted, padding=\"longest\", return_tensors=\"pt\").input_ids\n            if untruncated_ids.shape[-1] >= text_input_ids.shape[-1] and not torch.equal(\n                text_input_ids, untruncated_ids\n            ):\n                removed_text = tokenizer.batch_decode(untruncated_ids[:, max_seq_len - 1 : -1])\n                context.logger.warning(\n                    f\"The following part of your input was truncated because `max_sequence_length` is set to \"\n                    f\"{max_seq_len} tokens: {removed_text}\"\n                )\n\n            # Get hidden states from the text encoder\n            # Use the second-to-last hidden state like diffusers does\n            prompt_mask = attention_mask.to(device).bool()","sourceCodeStart":131,"sourceCodeEnd":167,"githubUrl":"https://github.com/invoke-ai/InvokeAI/blob/0b6a024f2ff6a86bfb953dcdb9cc504ef7397a06/invokeai/app/invocations/z_image_text_encoder.py#L131-L167","documentation":"Immediately after the input_ids check, _encode_prompt validates that text_inputs.attention_mask is a torch.Tensor. A non-tensor attention_mask raises a TypeError stating the tokenizer returned an unexpected type. The attention mask is required for the padded Qwen3 forward pass, so its type must be guaranteed.","triggerScenarios":"Z-Image text encoding where tokenizer(..., padding=\"longest\", return_tensors=\"pt\").attention_mask is not a torch.Tensor, e.g. returned as a list/ndarray by a custom tokenizer.","commonSituations":"Custom tokenizer subclass bypassing tensor conversion; transformers version mismatch; manually constructed BatchEncoding lacking tensorized fields.","solutions":["Use the stock transformers Qwen3 tokenizer so attention_mask is tensorized with return_tensors=\"pt\".","Upgrade transformers if return_tensors is being ignored.","Validate the tokenizer output before passing to the invocation (isinstance check on attention_mask).","Convert manually if required: attention_mask = torch.tensor(raw_mask)."],"exampleFix":"// before\ntext_inputs = tok(prompt)  # returns lists\n// after\ntext_inputs = tok(prompt, padding=\"longest\", return_tensors=\"pt\")  # tensors","handlingStrategy":"type-guard","validationCode":"inputs = tok(prompt, padding=\"longest\", return_tensors=\"pt\")\nif not isinstance(inputs.attention_mask, torch.Tensor):\n    fail_fast(inputs.attention_mask)","typeGuard":"def is_tensor(x) -> bool:\n    import torch\n    return isinstance(x, torch.Tensor)","tryCatchPattern":"try:\n    encode(context)\nexcept TypeError as e:\n    if \"Expected torch.Tensor for attention_mask\" in str(e):\n        swap_to_stock_tokenizer()\n    else:\n        raise","preventionTips":["Use stock transformers tokenizers so attention_mask is tensorized automatically.","Never construct BatchEncoding manually without tensor conversion.","Pin a transformers version known to work with your tokenizer files."],"tags":["type-check","tokenizer","attention-mask","z-image"],"backgroundTag":"unexpected-model-type","analyzedSha":"0b6a024f2ff6a86bfb953dcdb9cc504ef7397a06","analyzedAt":"2026-08-29T04:46:49.967Z","schemaVersion":2},"datasetVersion":"2026-08-29T07:17:48.351Z"}