{"record":{"id":"9ed4793bb0ecbd5f","repo":"huggingface/transformers","slug":"tensor-query-has-shape-with-a-zero-dimension-fla","errorCode":null,"errorMessage":"Tensor query has shape  with a zero dimension.\nFlashAttention does not support inputs with dim=0.\nPlease check your input shapes or use SDPA instead.","messagePattern":"Tensor query has shape  with a zero dimension\\.\nFlashAttention does not support inputs with dim=0\\.\nPlease check your input shapes or use SDPA instead\\.","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/transformers/integrations/flash_attention.py","lineNumber":50,"sourceCode":"    dropout: float = 0.0,\n    scaling: float | None = None,\n    sliding_window: int | None = None,\n    softcap: float | None = None,\n    is_causal: bool | None = None,\n    s_aux: torch.Tensor | None = None,  # alias: learnable attention sink\n    **kwargs,\n) -> tuple[torch.Tensor, None]:\n    if kwargs.get(\"output_attentions\", False):\n        logger.warning_once(\n            \"Flash Attention does not support `output_attentions=True`.\"\n            \" Please set your attention to `eager` if you want any of these features.\"\n        )\n\n    # This is before the transpose\n    seq_len = query.shape[2]\n\n    if any(dim == 0 for dim in query.shape):\n        raise ValueError(\n            \"Tensor query has shape  with a zero dimension.\\n\"\n            \"FlashAttention does not support inputs with dim=0.\\n\"\n            \"Please check your input shapes or use SDPA instead.\"\n        )\n    # FA2 uses non-transposed inputs\n    query = query.transpose(1, 2)\n    key = key.transpose(1, 2)\n    value = value.transpose(1, 2)\n\n    # FlashAttention requires the query and value to share a head dim; pad `value` up to the\n    # query head dim (e.g. MLA, where `v_head_dim < qk_head_dim`) and crop the output below.\n    head_dim, v_head_dim = query.shape[-1], value.shape[-1]\n    if v_head_dim != head_dim:\n        value = torch.nn.functional.pad(value, [0, head_dim - v_head_dim])\n\n    # In PEFT, usually we cast the layer norms in float32 for training stability reasons\n    # therefore the input hidden states gets silently casted in float32. Hence, we need\n    # cast them back in the correct dtype just to be sure everything works as expected.","sourceCodeStart":32,"sourceCodeEnd":68,"githubUrl":"https://github.com/huggingface/transformers/blob/a597f974857b3d92939971296bc0deb93d33d780/src/transformers/integrations/flash_attention.py#L32-L68","documentation":"The FlashAttention-2 integration (_flash_attention_forward) checks every dimension of the query tensor before transposing and calling into flash_attn. flash-attn kernels cannot operate on tensors with a 0-sized dimension (e.g. batch=0 or seq_len=0), so transformers raises ValueError up front with a suggestion to use SDPA, which tolerates empty batches.","triggerScenarios":"Calling a model with attn_implementation=\"flash_attention_2\" where the input produces a zero-dim query: batch_size=0, an empty padded batch, seq_length=0, or num_key_value_heads=0 due to config errors.","commonSituations":"DataLoader tail batches that are empty; speculative/lookahead decoding paths that occasionally build 0-length sequences; tests that pass dummy empty tensors; feeding preprocessed input_ids of shape [0, N].","solutions":["Guard the forward call: skip or short-circuit when query/input batch or sequence length is 0","Filter empty sequences out of the batch before the model call","Switch attention to \"sdpa\" (model.config._attn_implementation or attn_implementation kwarg) if empty batches must flow through"],"exampleFix":"# before\nout = model(input_ids)  # input_ids.shape == (0, 128) with flash_attention_2\n# ValueError: Tensor query has shape with a zero dimension\n\n# after\nif input_ids.shape[0] == 0 or input_ids.shape[-1] == 0:\n    out = None  # or skip batch\nelse:\n    out = model(input_ids)","handlingStrategy":"type-guard","validationCode":"def has_zero_dim(t):\n    return any(d == 0 for d in t.shape)\n\nif not has_zero_dim(query) and not has_zero_dim(key):\n    out = model(input_ids)  # flash_attention_2 safe","typeGuard":"def is_fa2_safe(batch) -> bool:\n    \"\"\"True when no tensor dimension is 0 (FlashAttention requirement).\"\"\"\n    return all(all(d != 0 for d in t.shape) for t in batch.values() if hasattr(t, \"shape\"))","tryCatchPattern":null,"preventionTips":["Drop empty batches/sequences in the DataLoader (filter len == 0) when using flash_attention_2","Fall back to attn_implementation=\"sdpa\" if your pipeline legitimately produces zero-sized batches"],"tags":["flash-attention","attention","empty-tensor","shape-validation"],"backgroundTag":null,"analyzedSha":"a597f974857b3d92939971296bc0deb93d33d780","analyzedAt":"2026-08-14T18:24:08.354Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}