dotnet/machinelearning · error · InvalidOperationException

Invalid state, either qkv_proj or q_proj, k_proj, v_proj sho

Error message

Invalid state, either qkv_proj or q_proj, k_proj, v_proj should be initialized

What it means

Attention.forward requires projection weights in one of two layouts: a fused qkv_proj, or separate q_proj/k_proj/v_proj. If neither is initialized (null), the layer cannot compute query/key/value states and throws InvalidOperationException.

Source

Thrown at src/Microsoft.ML.GenAI.Core/Module/Attention.cs:160

            Tensor valueStates;

            if (this.qkv_proj is not null)
            {
                var qkv = this.qkv_proj.forward(hiddenStates);
                var queryPos = this._numHeads * this._headDim;
                queryStates = qkv[.., .., ..queryPos];
                keyStates = qkv[.., .., queryPos..(queryPos + this._numKeyValueHeads * this._headDim)];
                valueStates = qkv[.., .., (queryPos + this._numKeyValueHeads * this._headDim)..];
            }
            else if (this.q_proj is not null && this.k_proj is not null && this.v_proj is not null)
            {
                queryStates = this.q_proj.forward(hiddenStates);
                keyStates = this.k_proj.forward(hiddenStates);
                valueStates = this.v_proj.forward(hiddenStates);
            }
            else
            {
                throw new InvalidOperationException("Invalid state, either qkv_proj or q_proj, k_proj, v_proj should be initialized");
            }

            queryStates = queryStates.view(bsz, qLen, this._numHeads, this._headDim).transpose(1, 2);
            keyStates = keyStates.view(bsz, qLen, this._numKeyValueHeads, this._headDim).transpose(1, 2);
            valueStates = valueStates.view(bsz, qLen, this._numKeyValueHeads, this._headDim).transpose(1, 2);
            var kvSeqLen = keyStates.IntShape()[^2];
            var pastKeyValue = input.Cache;
            if (pastKeyValue is not null)
            {
                kvSeqLen += pastKeyValue.GetUsableLength(kvSeqLen, this._layerIdx);
            }
            (queryStates, keyStates) = Utils.ApplyRotaryPosEmb(queryStates, keyStates, input.PositionalEmbeddings.Cos, input.PositionalEmbeddings.Sin);

            if (pastKeyValue is not null)
            {
                (keyStates, valueStates) = pastKeyValue.UpdateKVCache(keyStates, valueStates, this._layerIdx);
            }

View on GitHub (pinned to 7b76e69cf9)

Solutions

1. Ensure the checkpoint contains either qkv_proj or q_proj/k_proj/v_proj tensors and load them fully
2. Verify weight names in the state dict match what the loader expects (fused vs split naming)
3. Initialize the attention module with one of the two projection configurations before calling forward

Example fix

// before
var attn = new Attention(config); // no projections initialized
attn.forward(hiddenStates);
// after
var attn = new Attention(config);
attn.qkv_proj = loadedQkvProj; // or q_proj/k_proj/v_proj
attn.forward(hiddenStates);

bool projectionsReady = attn.qkv_proj is not null || (attn.q_proj is not null && attn.k_proj is not null && attn.v_proj is not null);

bool CanForward(Attention attn) => attn.qkv_proj is not null || (attn.q_proj is not null && attn.k_proj is not null && attn.v_proj is not null);

try { attn.forward(hiddenStates); } catch (InvalidOperationException ex) when (ex.Message.Contains("qkv_proj")) { LoadAttentionWeights(attn, checkpoint); }

When it happens

Trigger: Forward pass through an attention layer whose weights were loaded from a checkpoint missing qkv projections, or a model constructed with neither the fused nor the split projection fields initialized.

Common situations: Partial weight loading from a checkpoint format that names projections differently; manually constructing attention modules without calling weight initialization/loading; model config mismatch causing loaders to skip projection tensors.

Understand the failure class

Background: "Invalid state transition" errors: "status must be X, actually Y", "already rejected/charging/uninstalled", "cannot ... while running" — what they mean when a library rejects your call — this error's family across 31 libraries.

Related errors

• There are no columns in the DataFrame to use as value column
• Start must be called on a ModelLoader before it can be used.
• Current estimator chain has no estimator, can't append cache
• Failed to generate a reply.
• The layer count is not enough to cover all layers, did you f

AI-assisted analysis of dotnet/machinelearning@7b76e69cf9 (2026-09-11). Data as JSON: /api/errors/a25b51c3bae0bfba. Report an issue: GitHub.