{"record":{"id":"5d26cbba004499ae","repo":"dotnet/machinelearning","slug":"num-key-value-heads-must-be-specified","errorCode":null,"errorMessage":"num_key_value_heads must be specified","messagePattern":"num_key_value_heads must be specified","errorType":"validation","errorClass":"ArgumentException","httpStatus":null,"severity":"error","filePath":"src/Microsoft.ML.GenAI.Phi/Module/Phi2Attention.cs","lineNumber":64,"sourceCode":"    private readonly LayerNorm? k_layernorm;\n\n    private readonly Phi2RotaryEmbedding phiRotaryEmbedding;\n\n    // cache_k, cache_v\n    private Tensor cache_k;\n    private Tensor cache_v;\n#pragma warning restore MSML_PrivateFieldName // Private field name not in: _camelCase format\n\n    public Phi2Attention(Phi2Config config, int? layerIdx = null, int maxBatch = 2, int maxLength = 1024)\n        : base(nameof(Phi2Attention))\n    {\n        this._layerIdx = layerIdx;\n        this._config = config;\n        this._attentionDropout = config.AttentionDropout;\n        this._hiddenSize = config.HiddenSize;\n        this._numAttentionHeads = config.NumAttentionHeads;\n        this._headDim = this._hiddenSize / this._numAttentionHeads;\n        this._numKeyValueHeads = config.NumKeyValueHeads ?? throw new ArgumentException(\"num_key_value_heads must be specified\");\n        this._numKeyValueGroups = this._numAttentionHeads / this._numKeyValueHeads;\n        this._maxPositionEmbeddings = config.MaxPositionEmbeddings;\n        this._ropeTheta = config.RopeTheta;\n        this._partialRotaryFactor = config.PartialRotaryFactor;\n\n        Contract.Assert(this._hiddenSize % (this._headDim * this._numAttentionHeads) == 0, \"hidden_size must be divisible by num_attention_heads\");\n        this.q_proj = new GenAILinear(this._hiddenSize, this._numAttentionHeads * this._headDim, hasBias: true, dtype: config.Dtype);\n        this.k_proj = new GenAILinear(this._hiddenSize, this._numKeyValueHeads * this._headDim, hasBias: true, dtype: config.Dtype);\n        this.v_proj = new GenAILinear(this._hiddenSize, this._numKeyValueHeads * this._headDim, hasBias: true, dtype: config.Dtype);\n        this.dense = new GenAILinear(this._numAttentionHeads * this._headDim, this._hiddenSize, hasBias: true, dtype: config.Dtype);\n\n        this._qkLayernorm = config.QkLayernorm;\n        if (this._qkLayernorm)\n        {\n            this.q_layernorm = nn.LayerNorm(this._hiddenSize / this._numAttentionHeads, eps: config.LayerNormEps, elementwise_affine: true, dtype: config.Dtype);\n            this.k_layernorm = nn.LayerNorm(this._hiddenSize / this._numAttentionHeads, eps: config.LayerNormEps, elementwise_affine: true, dtype: config.Dtype);\n        }\n","sourceCodeStart":46,"sourceCodeEnd":82,"githubUrl":"https://github.com/dotnet/machinelearning/blob/7b76e69cf964daeca3f1377af6bc5543284d56c6/src/Microsoft.ML.GenAI.Phi/Module/Phi2Attention.cs#L46-L82","documentation":"Phi2Attention computes grouped-query attention and needs NumKeyValueHeads from the Phi2 config. If config.NumKeyValueHeads is null (not set when loading a checkpoint/config), the constructor throws ArgumentException('num_key_value_heads must be specified').","triggerScenarios":"Constructing Phi2Attention (e.g. via Phi2Model or CreateAttentionFromConfig) with a config whose num_key_value_heads field is absent — common with original Phi-2 checkpoints that predate GQA fields.","commonSituations":"Loading a HuggingFace Phi-2 config.json lacking num_key_value_heads; hand-building a Phi2Config without setting NumKeyValueHeads; newer library version requiring GQA where older configs didn't include it.","solutions":["Set NumKeyValueHeads in the config (for Phi-2, equal to NumAttentionHeads)","Patch the model's config.json to add \"num_key_value_heads\" before loading","Default it explicitly when building config programmatically: NumKeyValueHeads = NumAttentionHeads"],"exampleFix":"// before\nvar config = new Phi2Config { HiddenSize = 2560, NumAttentionHeads = 32 };\n// after\nvar config = new Phi2Config { HiddenSize = 2560, NumAttentionHeads = 32, NumKeyValueHeads = 32 };","handlingStrategy":"validation","validationCode":"if (config.NumKeyValueHeads is null) config.NumKeyValueHeads = config.NumAttentionHeads;","typeGuard":"static bool IsPhi2ConfigComplete(Phi2Config c) => c.NumKeyValueHeads.HasValue && c.NumAttentionHeads > 0;","tryCatchPattern":"try { model = new Phi2Model(config); }\ncatch (ArgumentException ex) when (ex.Message.Contains(\"num_key_value_heads\")) { config.NumKeyValueHeads = config.NumAttentionHeads; model = new Phi2Model(config); }","preventionTips":["Validate required GQA fields right after loading any checkpoint config","Patch legacy Phi-2 config.json to include num_key_value_heads","Centralize config normalization in one loader function"],"tags":["csharp","phi-2","config"],"backgroundTag":"missing-required-config-field","analyzedSha":"7b76e69cf964daeca3f1377af6bc5543284d56c6","analyzedAt":"2026-09-11T12:35:38.930Z","contentChangedAt":"2026-09-11T12:35:38.930Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}