huggingface/transformers · error · ValueError
`attention_mask` passed to `generate` must be 2D.
Error message
`attention_mask` passed to `generate` must be 2D.
What it means
Error "`attention_mask` passed to `generate` must be 2D." thrown in huggingface/transformers.
Source
Thrown at src/transformers/generation/utils.py:2562
logger.warning(
"A decoder-only architecture is being used, but right-padding was detected! For correct "
"generation results, please set `padding_side='left'` when initializing the tokenizer."
)
# 4. Define other model kwargs
# decoder-only models with inputs_embeds forwarding must use caching (otherwise we can't detect whether we are
# generating the first new token or not, and we only want to use the embeddings for the first new token)
if not self.config.is_encoder_decoder and model_input_name == "inputs_embeds":
generation_config.use_cache = True
if not kwargs_has_attention_mask and not self.config.is_encoder_decoder and accepts_attention_mask:
model_kwargs["attention_mask"] = self._prepare_attention_mask_for_generation(
inputs_tensor, generation_config, model_kwargs
)
elif kwargs_has_attention_mask:
# TODO (joao): generalize this check with other types of inputs
if model_input_name == "input_ids" and len(model_kwargs["attention_mask"].shape) > 2:
raise ValueError("`attention_mask` passed to `generate` must be 2D.")
kwargs_has_position_ids = model_kwargs.get("position_ids", None) is not None
accepts_position_ids = "position_ids" in set(inspect.signature(self.forward).parameters.keys())
if not kwargs_has_position_ids and accepts_position_ids and not self.config.is_encoder_decoder:
model_kwargs["position_ids"] = self._prepare_position_ids_for_generation(inputs_tensor, model_kwargs)
if self.config.is_encoder_decoder and "encoder_outputs" not in model_kwargs:
# if model is encoder decoder encoder_outputs are created and added to `model_kwargs`
model_kwargs = self._prepare_encoder_decoder_kwargs_for_generation(
inputs_tensor, model_kwargs, model_input_name, generation_config
)
# 5. Prepare `input_ids` which will be used for auto-regressive generation
if self.config.is_encoder_decoder:
input_ids, model_kwargs = self._prepare_decoder_input_ids_for_generation(
batch_size=batch_size,
model_input_name=model_input_name,
model_kwargs=model_kwargs,View on GitHub (pinned to a597f97485)
Solutions
- Pass a 2D `attention_mask` of shape (batch_size, sequence_length).
- Expand a 1D mask with `mask.unsqueeze(0)` for batch size 1.
When it happens
Trigger: Raised in generate() input preparation when the attention_mask tensor has a rank other than 2.
Common situations: Passing a 1D or 3D attention_mask to generate(), often from manual tensor construction instead of tokenizer output.
AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14).
Data as JSON: /api/errors/fdfcd0a665623992.
Report an issue: GitHub.