{"record":{"id":"969836899dabf946","repo":"dotnet/machinelearning","slug":"the-layer-count-is-not-enough-to-cover-all-layers","errorCode":null,"errorMessage":"The layer count is not enough to cover all layers, did you forget to set the last layer count to -1?","messagePattern":"The layer count is not enough to cover all layers, did you forget to set the last layer count to -1\\?","errorType":"validation","errorClass":"ArgumentException","httpStatus":null,"severity":"error","filePath":"src/Microsoft.ML.GenAI.Core/Extension/ModuleExtension.cs","lineNumber":245,"sourceCode":"                {\n                    deviceMap[key] = device;\n                }\n            }\n            else\n            {\n                foreach (var (key, value) in layerSizeMap)\n                {\n                    deviceMap[key] = device;\n                }\n\n                layerSizeMap.Clear();\n                break;\n            }\n        }\n\n        if (layerSizeMap.Count > 0)\n        {\n            throw new ArgumentException(\"The layer count is not enough to cover all layers, did you forget to set the last layer count to -1?\");\n        }\n\n        return deviceMap;\n    }\n\n    internal static string Peek(this nn.Module model)\n    {\n        var sb = new StringBuilder();\n        var stateDict = model.state_dict();\n        // preview state_dict\n        int i = 0;\n        foreach (var (key, value) in stateDict.OrderBy(x => x.Key, StringComparer.OrdinalIgnoreCase))\n        {\n            var str = value.Peek(key);\n            sb.AppendLine($\"{i}: {str}\");\n            i++;\n        }\n","sourceCodeStart":227,"sourceCodeEnd":263,"githubUrl":"https://github.com/dotnet/machinelearning/blob/7b76e69cf964daeca3f1377af6bc5543284d56c6/src/Microsoft.ML.GenAI.Core/Extension/ModuleExtension.cs#L227-L263","documentation":"When building a device map for a model, InferDeviceMapForEachLayer maps each layer by size using a layerSizeMap whose last entry must be -1 (meaning 'remaining layers / rest of weights'). If after inference any entries in the map remain unresolved (layerSizeMap.Count > 0), the method assumes the caller forgot to mark the final layer count as -1 and throws ArgumentException.","triggerScenarios":"Calling InferDeviceMapForEachLayer (e.g. to prepare llama for multi-device inference) with a device map configuration where no layer entry has count -1, or where declared layer counts do not sum to the model's actual layer count.","commonSituations":"Hand-written device map configs copied from a differently-sized model checkpoint (e.g. 7B map used for 13B); typo where the last line uses the actual layer count instead of -1.","solutions":["Set the last layer's count to -1 in the device map so it consumes all remaining layers","Verify declared layer counts match the checkpoint's actual number of transformer blocks","Regenerate the device map from the model instead of reusing a config from another model size"],"exampleFix":"// before\nvar deviceMap = model.InferDeviceMapForEachLayer(metadata, new Dictionary<string, long> {\n  [\"model.embed_tokens\"] = 0,\n  [\"model.layers.0\"] = 32,\n  [\"lm_head\"] = 1\n});\n// after\nvar deviceMap = model.InferDeviceMapForEachLayer(metadata, new Dictionary<string, long> {\n  [\"model.embed_tokens\"] = 0,\n  [\"model.layers.0\"] = 32,\n  [\"lm_head\"] = -1\n});","handlingStrategy":"validation","validationCode":"bool hasRestEntry = deviceMapEntries.Values.Contains(-1);\nif (!hasRestEntry) throw new ArgumentException(\"Device map must set the last layer count to -1\");","typeGuard":null,"tryCatchPattern":"try { deviceMap = model.InferDeviceMapForEachLayer(metadata, entries); } catch (ArgumentException ex) when (ex.Message.Contains(\"layer count\")) { entries[entries.Keys.Last()] = -1; deviceMap = model.InferDeviceMapForEachLayer(metadata, entries); }","preventionTips":["Always end device map layer counts with -1","Copy device maps only between models of identical architecture/size","Validate layer counts against checkpoint metadata before mapping"],"tags":["genai","device-map","configuration"],"backgroundTag":"invalid-config-value","analyzedSha":"7b76e69cf964daeca3f1377af6bc5543284d56c6","analyzedAt":"2026-09-11T12:35:38.930Z","contentChangedAt":"2026-09-11T12:35:38.930Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}