{"record":{"id":"5c71f823461fefc6","repo":"dotnet/machinelearning","slug":"dimension-must-be-divisible-by-2-5c71f8","errorCode":null,"errorMessage":"Dimension must be divisible by 2","messagePattern":"Dimension must be divisible by 2","errorType":"exception","errorClass":"ArgumentException","httpStatus":null,"severity":"error","filePath":"src/Microsoft.ML.GenAI.Phi/Utils.cs","lineNumber":25,"sourceCode":"using System.Linq;\nusing System.Reflection;\nusing System.Text;\nusing System.Threading.Tasks;\nusing TorchSharp;\nusing TorchSharp.Modules;\nusing static TorchSharp.torch;\nusing static TorchSharp.torch.nn;\nnamespace Microsoft.ML.GenAI.Phi;\n\ninternal static class Utils\n{\n    public static Tensor PrecomputeThetaPosFrequencies(int headDim, int seqLen, string device, float theta = 10000.0f)\n    {\n        // As written in the paragraph 3.2.2 of the paper\n        // >> In order to generalize our results in 2D to any xi ∈ Rd where **d is even**, [...]\n        if (headDim % 2 != 0)\n        {\n            throw new ArgumentException(\"Dimension must be divisible by 2\", nameof(headDim));\n        }\n\n        // Build the theta parameter\n        // According to the formula theta_i = 10000^(-2(i-1)/dim) for i = [1, 2, ... dim/2]\n        // Shape: (Head_Dim / 2)\n        var thetaNumerator = torch.arange(0, headDim, 2).to(torch.float32).to(device);\n        // Shape: (Head_Dim / 2)\n        var thetaInput = torch.pow(theta, -1.0f * (thetaNumerator / headDim)).to(device); // (Dim / 2)\n        // Construct the positions (the \"m\" parameter)\n        // Shape: (Seq_Len)\n        var m = torch.arange(seqLen, device: device);\n        // Multiply each theta by each position using the outer product.\n        // Shape: (Seq_Len) outer_product* (Head_Dim / 2) -> (Seq_Len, Head_Dim / 2)\n        var freqs = torch.outer(m, thetaInput).to(torch.float32).to(device);\n\n        // We can compute complex numbers in the polar form c = R * exp(m * theta), where R = 1 as follows:\n        // (Seq_Len, Head_Dim / 2) -> (Seq_Len, Head_Dim / 2)\n        var freqsComplex = torch.polar(torch.ones_like(freqs), freqs);","sourceCodeStart":7,"sourceCodeEnd":43,"githubUrl":"https://github.com/dotnet/machinelearning/blob/7b76e69cf964daeca3f1377af6bc5543284d56c6/src/Microsoft.ML.GenAI.Phi/Utils.cs#L7-L43","documentation":"PrecomputeThetaPosFrequencies computes RoPE (rotary position embedding) frequency tables for the Phi model. The paper's formula theta_i = 10000^(-2(i-1)/dim) requires an even head dimension so frequencies can be built in headDim/2 pairs, so the method rejects odd headDim with an ArgumentException naming the headDim parameter.","triggerScenarios":"Calling PrecomputeThetaPosFrequencies with a headDim argument that is not divisible by 2, e.g. passing a hidden size, embedding size, or an odd model config value instead of the per-head dimension.","commonSituations":"Hand-configuring a custom Phi/LLaMA-style model where head_dim was changed or derived incorrectly (e.g. total dim / heads yielding an odd number); porting weights from a nonstandard checkpoint.","solutions":["Pass the per-head attention dimension (must be even, e.g. 64, 80, 128), not the total hidden size","Verify model config: headDim = hiddenSize / numAttentionHeads and confirm it is even","If a custom checkpoint truly has odd headDim, this RoPE implementation cannot be used as-is; pad or use a different implementation"],"exampleFix":"// before\nTensor freqs = PrecomputeThetaPosFrequencies(modelConfig.HiddenSize, seqLen, device);\n// after\nint headDim = modelConfig.HiddenSize / modelConfig.NumAttentionHeads; // e.g. 4096/32 = 128\nTensor freqs = PrecomputeThetaPosFrequencies(headDim, seqLen, device);","handlingStrategy":"validation","validationCode":"if (headDim % 2 != 0)\n    throw new ArgumentException($\"headDim must be even for RoPE; got {headDim}\", nameof(headDim));","typeGuard":"bool IsValidHeadDim(int headDim) => headDim > 0 && headDim % 2 == 0;","tryCatchPattern":"try\n{\n    freqs = PrecomputeThetaPosFrequencies(headDim, seqLen, device);\n}\ncatch (ArgumentException ex) when (ex.ParamName == \"headDim\")\n{\n    logger.LogError(ex, \"Invalid headDim {HeadDim} for RoPE\", headDim);\n}","preventionTips":["Derive headDim as hiddenSize / numAttentionHeads, never pass hiddenSize directly","Assert evenness of headDim at model-config load time","Add a unit test per model config before training/inference runs"],"tags":["csharp","machine-learning","rope","argument-validation"],"backgroundTag":"invalid-argument-value","analyzedSha":"7b76e69cf964daeca3f1377af6bc5543284d56c6","analyzedAt":"2026-09-11T12:35:38.930Z","contentChangedAt":"2026-09-11T12:35:38.930Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}