dotnet/machinelearning · error · ArgumentException

num_key_value_heads must be specified

Error message

num_key_value_heads must be specified

What it means

Phi3DecoderLayer.CreateAttentionFromConfig builds the attention module using config.NumKeyValueHeads. When NumKeyValueHeads is null the null-coalescing throw raises ArgumentException('num_key_value_heads must be specified'); Phi-3 requires this field for its GQA layout.

Source

Thrown at src/Microsoft.ML.GenAI.Phi/Module/Phi3DecoderLayer.cs:149

        hiddenStates = this.mlp.forward(hiddenStates);
        hiddenStates = residual + this.resid_mlp_dropout.forward(hiddenStates);

        if (UnloadFromDeviceFunc != null)
        {
            UnloadFromDeviceFunc(this);
        }
        return new Phi3DecoderLayerOutput(hiddenStates.MoveToOuterDisposeScope(), selfAttnWeights?.MoveToOuterDisposeScope(), presentKeyValue);
    }

    private Attention CreateAttentionFromConfig(Phi3Config config, int layerIdx)
    {
        var headDim = config.HiddenSize / config.NumAttentionHeads;
        return new Attention(
            attentionDropout: config.AttentionDropout,
            hiddenSize: config.HiddenSize,
            numHeads: config.NumAttentionHeads,
            headDim: headDim,
            numKeyValueHeads: config.NumKeyValueHeads ?? throw new ArgumentException("num_key_value_heads must be specified"),
            numKeyValueGroups: config.NumAttentionHeads / config.NumKeyValueHeads ?? throw new ArgumentException("num_key_value_heads must be specified"),
            maxPositionEmbeddings: config.MaxPositionEmbeddings,
            originalMaxPositionEmbeddings: config.OriginalMaxPositionEmbeddings,
            layerIdx: layerIdx,
            useQkvProj: true,
            dtype: config.DType);
    }
}

View on GitHub (pinned to 7b76e69cf9)

Solutions

1. Set NumKeyValueHeads in the config (typically equal to NumAttentionHeads for Phi-3)
2. Add "num_key_value_heads" to the model's config.json before loading
3. Validate the config right after load and fill defaults: if null, set to NumAttentionHeads

Example fix

// before
var config = Phi3Config.FromFile("config.json");
// after
var config = Phi3Config.FromFile("config.json"); config.NumKeyValueHeads ??= config.NumAttentionHeads;

if (config.NumKeyValueHeads is null) config.NumKeyValueHeads = config.NumAttentionHeads;

static bool HasGqaFields(Phi3Config c) => c.NumKeyValueHeads is int kv && kv > 0 && c.NumAttentionHeads % kv == 0;

try { var layer = new Phi3DecoderLayer(config, i); }
catch (ArgumentException ex) when (ex.Message.Contains("num_key_value_heads")) { config.NumKeyValueHeads = config.NumAttentionHeads; var layer = new Phi3DecoderLayer(config, i); }

When it happens

Trigger: Creating a Phi3DecoderLayer (eager path) with a config.json missing num_key_value_heads, or a programmatically built Phi3Config that left NumKeyValueHeads unset.

Common situations: Original Phi-3 checkpoints/exports predating the field; hand-written configs; conversion tools dropping unknown keys.

Understand the failure class

Background: "is required", "must be set", "missing required field": configuration validation errors across open-source libraries — this error's family across 36 libraries.

Related errors

• num_key_value_heads must be specified
• Only one of input_ids or inputs_embeds may be set
• No Onnx Session Options
• Activation function {name} not supported.
• {nameof(ChannelMessageKind)}.{e.Kind} is not yet implemented

AI-assisted analysis of dotnet/machinelearning@7b76e69cf9 (2026-09-11). Data as JSON: /api/errors/b2947f34003e33a7. Report an issue: GitHub.