{"record":{"id":"67ddc776499b378e","repo":"dotnet/machinelearning","slug":"attention-mask-should-be-2d","errorCode":null,"errorMessage":"Attention mask should be 2D","messagePattern":"Attention mask should be 2D","errorType":"validation","errorClass":"ArgumentException","httpStatus":null,"severity":"error","filePath":"src/Microsoft.ML.GenAI.Core/Utility/AttentionMaskConverter.cs","lineNumber":156,"sourceCode":"    /// <param name=\"slidingWindow\">The sliding window size.</param>\n    /// <param name=\"inputShape\">The input shape should be a tuple that defines `(batch_size, query_length)`.</param>\n    public static Tensor? Create4DCausalAttentionMask(\n        Tensor? attentionMask,\n        long[] inputShape,\n        ScalarType dType,\n        Device device,\n        int pastKeyValuesLength = 0,\n        int? slidingWindow = null)\n    {\n        var converter = new AttentionMaskConverter(isCausal: true, slidingWindow: slidingWindow);\n        var batchSize = (int)inputShape[0];\n        var queryLength = (int)inputShape[1];\n        var keyValueLength = pastKeyValuesLength + queryLength;\n        if (attentionMask is not null)\n        {\n            if (attentionMask.ndim != 2)\n            {\n                throw new ArgumentException(\"Attention mask should be 2D\");\n            }\n            return converter.To4D(attentionMask, (int)inputShape[1], dType, keyValueLength);\n        }\n\n        return converter.ToCausal4D(batchSize, queryLength, keyValueLength, dType, device);\n    }\n\n    public static Tensor ExpandMask(\n        Tensor mask,\n        ScalarType dType,\n        int? tgtLen = null)\n    {\n        var bsz = (int)mask.shape[0];\n        var srcLen = (int)mask.shape[1];\n        tgtLen ??= srcLen;\n\n        var expandedMask = mask.unsqueeze(1).unsqueeze(1).expand(bsz, 1, tgtLen.Value, srcLen).to(dType);\n        var invertedMask = 1.0 - expandedMask;","sourceCodeStart":138,"sourceCodeEnd":174,"githubUrl":"https://github.com/dotnet/machinelearning/blob/7b76e69cf964daeca3f1377af6bc5543284d56c6/src/Microsoft.ML.GenAI.Core/Utility/AttentionMaskConverter.cs#L138-L174","documentation":"AttentionMaskConverter.Create4DCausalAttentionMask throws ArgumentException when the supplied attentionMask tensor is not 2D. The converter expects a [batch, seq_len] padding mask which it expands to [batch, 1, tgt_len, src_len]; higher- or lower-rank inputs cannot be expanded.","triggerScenarios":"Passing an already-expanded 4D mask, a 3D per-head mask, or a flattened 1D mask into Create4DCausalAttentionMask.","commonSituations":"Feeding an output mask from a previous layer back in; reshaping mistakes when batching; porting PyTorch code that keeps masks 4D throughout, while this API wants the raw 2D mask each call.","solutions":["Pass the tokenizer/model output attention_mask directly — it is already [batch, seq_len] 2D.","If your mask is 4D, use it as-is without re-expanding (skip Create4DCausalAttentionMask).","Reshape 1D masks to 2D with mask.unsqueeze(0) (adding batch dimension) before the call.","Assert mask.ndim == 2 in your forward code before invoking the converter."],"exampleFix":"// before\nvar mask4d = AttentionMaskConverter.Create4DCausalAttentionMask(mask4dFromPreviousStep, inputShape, dType, kvLen, device);\n// after\nvar mask2d = attentionMaskFromTokenizer; // [batch, seq_len]\nvar mask4d = AttentionMaskConverter.Create4DCausalAttentionMask(mask2d, inputShape, dType, kvLen, device);","handlingStrategy":"validation","validationCode":"if (attentionMask is not null && attentionMask.ndim != 2)\n    throw new InvalidOperationException($\"attention_mask must be [batch, seq_len] 2D, got ndim={attentionMask.ndim}\");","typeGuard":"bool Is2DMask(Tensor? m) => m is not null && m.ndim == 2;","tryCatchPattern":"try { var mask = AttentionMaskConverter.Create4DCausalAttentionMask(attnMask, inputShape, dType, kvLen, device); }\ncatch (ArgumentException ex) when (ex.Message.Contains(\"should be 2D\")) { logger.LogError(\"Pass the raw tokenizer 2D attention_mask, not an expanded one\"); throw; }","preventionTips":["Pass the tokenizer output attention_mask directly; never a previously expanded 4D mask.","Unsqueeze 1D masks to add the batch dimension before the call.","Skip Create4DCausalAttentionMask if you already hold a 4D mask.","Assert mask shape in debug builds."],"tags":["attention","mask","tensor-shape"],"backgroundTag":"tensor-shape-mismatch","analyzedSha":"7b76e69cf964daeca3f1377af6bc5543284d56c6","analyzedAt":"2026-09-11T12:35:38.930Z","contentChangedAt":"2026-09-11T12:35:38.930Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}