{"record":{"id":"a1376305562a73f7","repo":"huggingface/candle","slug":"wrong-shape-for-input-ids-or-attention-mask","errorCode":null,"errorMessage":"Wrong shape for input_ids or attention_mask","messagePattern":"Wrong shape for input_ids or attention_mask","errorType":"validation","errorClass":null,"httpStatus":null,"severity":"error","filePath":"candle-transformers/src/models/bert.rs","lineNumber":519,"sourceCode":"        let _enter = self.span.enter();\n        let embedding_output = self.embeddings.forward(input_ids, token_type_ids)?;\n        let attention_mask = match attention_mask {\n            Some(attention_mask) => attention_mask.clone(),\n            None => input_ids.ones_like()?,\n        };\n        let dtype = embedding_output.dtype();\n        // https://github.com/huggingface/transformers/blob/6eedfa6dd15dc1e22a55ae036f681914e5a0d9a1/src/transformers/models/bert/modeling_bert.py#L995\n        let attention_mask = get_extended_attention_mask(&attention_mask, dtype)?;\n        let sequence_output = self.encoder.forward(&embedding_output, &attention_mask)?;\n        Ok(sequence_output)\n    }\n}\n\nfn get_extended_attention_mask(attention_mask: &Tensor, dtype: DType) -> Result<Tensor> {\n    let attention_mask = match attention_mask.rank() {\n        3 => attention_mask.unsqueeze(1)?,\n        2 => attention_mask.unsqueeze(1)?.unsqueeze(1)?,\n        _ => candle::bail!(\"Wrong shape for input_ids or attention_mask\"),\n    };\n    let attention_mask = attention_mask.to_dtype(dtype)?;\n    // torch.finfo(dtype).min\n    (attention_mask.ones_like()? - &attention_mask)?.broadcast_mul(\n        &Tensor::try_from(f32::MIN)?\n            .to_device(attention_mask.device())?\n            .to_dtype(dtype)?,\n    )\n}\n\n//https://github.com/huggingface/transformers/blob/1bd604d11c405dfb8b78bda4062d88fc75c17de0/src/transformers/models/bert/modeling_bert.py#L752-L766\nstruct BertPredictionHeadTransform {\n    dense: Linear,\n    activation: HiddenActLayer,\n    layer_norm: LayerNorm,\n}\n\nimpl BertPredictionHeadTransform {","sourceCodeStart":501,"sourceCodeEnd":537,"githubUrl":"https://github.com/huggingface/candle/blob/d5fee525bfde3273eb7c9b75fd2bc4937be867ca/candle-transformers/src/models/bert.rs#L501-L537","documentation":"BERT's get_extended_attention_mask only accepts attention masks of rank 2 (batch x seq_len) or rank 3, extending them for multi-head attention. Any other rank fails with this message, indicating the input_ids/attention_mask were built with an unexpected shape.","triggerScenarios":"Passing an attention_mask of rank 1 (single sequence, no batch dim), rank 4, or a mask already expanded to per-head shape into BertModel with attention masks.","commonSituations":"Preprocessing producing an unbatched 1-D mask, frameworks that already return [b, heads, seq, seq] masks, or accidentally swapping input_ids and attention_mask arguments.","solutions":["Ensure the mask is 2-D [batch, seq_len] (or 3-D) before calling the model","Add a batch dimension: mask.unsqueeze(0) for a single sequence","Verify argument order so a mask isn't passed where input_ids is expected","Squeeze extra trailing dims if the mask is already 4-D from another library"],"exampleFix":"// before\nlet mask = Tensor::ones((128, DType::U8)...)  // rank 1\nlet out = model.forward(&ids, &mask, ...)?;\n// after\nlet mask = Tensor::ones((1, 128, DType::U8)...).unsqueeze(0)?; // rank 2/3\nlet out = model.forward(&ids, &mask, ...)?;","handlingStrategy":"validation","validationCode":"let rank = attention_mask.rank();\nassert!(rank == 2 || rank == 3, \"attention_mask rank {} unsupported; expected 2 or 3\", rank);","typeGuard":"fn is_valid_attention_mask(mask: &Tensor) -> bool {\n    matches!(mask.rank(), 2 | 3)\n}","tryCatchPattern":"let mask = match attention_mask.rank() {\n    2 | 3 => attention_mask,\n    1 => attention_mask.unsqueeze(0)?,\n    _ => return Err(anyhow!(\"unsupported mask rank\")),\n};\nlet out = model.forward(&input_ids, &mask, &token_type_ids, None, None)?;","preventionTips":["Keep masks batched: always [batch, seq_len]","Check mask rank before calling the model","Don't pass already-expanded 4-D masks from other frameworks","Verify you're not swapping input_ids and attention_mask arguments"],"tags":["rust","candle","bert","attention-mask","shape-mismatch"],"backgroundTag":"tensor-rank-mismatch","analyzedSha":"d5fee525bfde3273eb7c9b75fd2bc4937be867ca","analyzedAt":"2026-09-02T00:15:47.023Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-09T06:17:21.866Z"}