{"record":{"id":"a25b51c3bae0bfba","repo":"dotnet/machinelearning","slug":"invalid-state-either-qkv-proj-or-q-proj-k-proj","errorCode":null,"errorMessage":"Invalid state, either qkv_proj or q_proj, k_proj, v_proj should be initialized","messagePattern":"Invalid state, either qkv_proj or q_proj, k_proj, v_proj should be initialized","errorType":"exception","errorClass":"InvalidOperationException","httpStatus":null,"severity":"error","filePath":"src/Microsoft.ML.GenAI.Core/Module/Attention.cs","lineNumber":160,"sourceCode":"            Tensor valueStates;\n\n            if (this.qkv_proj is not null)\n            {\n                var qkv = this.qkv_proj.forward(hiddenStates);\n                var queryPos = this._numHeads * this._headDim;\n                queryStates = qkv[.., .., ..queryPos];\n                keyStates = qkv[.., .., queryPos..(queryPos + this._numKeyValueHeads * this._headDim)];\n                valueStates = qkv[.., .., (queryPos + this._numKeyValueHeads * this._headDim)..];\n            }\n            else if (this.q_proj is not null && this.k_proj is not null && this.v_proj is not null)\n            {\n                queryStates = this.q_proj.forward(hiddenStates);\n                keyStates = this.k_proj.forward(hiddenStates);\n                valueStates = this.v_proj.forward(hiddenStates);\n            }\n            else\n            {\n                throw new InvalidOperationException(\"Invalid state, either qkv_proj or q_proj, k_proj, v_proj should be initialized\");\n            }\n\n            queryStates = queryStates.view(bsz, qLen, this._numHeads, this._headDim).transpose(1, 2);\n            keyStates = keyStates.view(bsz, qLen, this._numKeyValueHeads, this._headDim).transpose(1, 2);\n            valueStates = valueStates.view(bsz, qLen, this._numKeyValueHeads, this._headDim).transpose(1, 2);\n            var kvSeqLen = keyStates.IntShape()[^2];\n            var pastKeyValue = input.Cache;\n            if (pastKeyValue is not null)\n            {\n                kvSeqLen += pastKeyValue.GetUsableLength(kvSeqLen, this._layerIdx);\n            }\n            (queryStates, keyStates) = Utils.ApplyRotaryPosEmb(queryStates, keyStates, input.PositionalEmbeddings.Cos, input.PositionalEmbeddings.Sin);\n\n            if (pastKeyValue is not null)\n            {\n                (keyStates, valueStates) = pastKeyValue.UpdateKVCache(keyStates, valueStates, this._layerIdx);\n            }\n","sourceCodeStart":142,"sourceCodeEnd":178,"githubUrl":"https://github.com/dotnet/machinelearning/blob/7b76e69cf964daeca3f1377af6bc5543284d56c6/src/Microsoft.ML.GenAI.Core/Module/Attention.cs#L142-L178","documentation":"Attention.forward requires projection weights in one of two layouts: a fused qkv_proj, or separate q_proj/k_proj/v_proj. If neither is initialized (null), the layer cannot compute query/key/value states and throws InvalidOperationException.","triggerScenarios":"Forward pass through an attention layer whose weights were loaded from a checkpoint missing qkv projections, or a model constructed with neither the fused nor the split projection fields initialized.","commonSituations":"Partial weight loading from a checkpoint format that names projections differently; manually constructing attention modules without calling weight initialization/loading; model config mismatch causing loaders to skip projection tensors.","solutions":["Ensure the checkpoint contains either qkv_proj or q_proj/k_proj/v_proj tensors and load them fully","Verify weight names in the state dict match what the loader expects (fused vs split naming)","Initialize the attention module with one of the two projection configurations before calling forward"],"exampleFix":"// before\nvar attn = new Attention(config); // no projections initialized\nattn.forward(hiddenStates);\n// after\nvar attn = new Attention(config);\nattn.qkv_proj = loadedQkvProj; // or q_proj/k_proj/v_proj\nattn.forward(hiddenStates);","handlingStrategy":"type-guard","validationCode":"bool projectionsReady = attn.qkv_proj is not null || (attn.q_proj is not null && attn.k_proj is not null && attn.v_proj is not null);","typeGuard":"bool CanForward(Attention attn) => attn.qkv_proj is not null || (attn.q_proj is not null && attn.k_proj is not null && attn.v_proj is not null);","tryCatchPattern":"try { attn.forward(hiddenStates); } catch (InvalidOperationException ex) when (ex.Message.Contains(\"qkv_proj\")) { LoadAttentionWeights(attn, checkpoint); }","preventionTips":["Load full checkpoint weights before any forward pass","Confirm checkpoint naming (fused vs split qkv) matches loader expectations","Assert projections non-null in model setup code"],"tags":["genai","attention","invalid-state"],"backgroundTag":"invalid-state-transition","analyzedSha":"7b76e69cf964daeca3f1377af6bc5543284d56c6","analyzedAt":"2026-09-11T12:35:38.930Z","contentChangedAt":"2026-09-11T12:35:38.930Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}