dotnet/machinelearning · error · ArgumentException

The layer count is not enough to cover all layers, did you…

Error message

The layer count is not enough to cover all layers, did you forget to set the last layer count to -1?

What it means

When building a device map for a model, InferDeviceMapForEachLayer maps each layer by size using a layerSizeMap whose last entry must be -1 (meaning 'remaining layers / rest of weights'). If after inference any entries in the map remain unresolved (layerSizeMap.Count > 0), the method assumes the caller forgot to mark the final layer count as -1 and throws ArgumentException.

Solutions

Set the last layer's count to -1 in the device map so it consumes all remaining layers
Verify declared layer counts match the checkpoint's actual number of transformer blocks
Regenerate the device map from the model instead of reusing a config from another model size

Example fix

// before
var deviceMap = model.InferDeviceMapForEachLayer(metadata, new Dictionary<string, long> {
  ["model.embed_tokens"] = 0,
  ["model.layers.0"] = 32,
  ["lm_head"] = 1
});
// after
var deviceMap = model.InferDeviceMapForEachLayer(metadata, new Dictionary<string, long> {
  ["model.embed_tokens"] = 0,
  ["model.layers.0"] = 32,
  ["lm_head"] = -1
});

Defensive patterns

Strategy: validation

Validate before calling

bool hasRestEntry = deviceMapEntries.Values.Contains(-1);
if (!hasRestEntry) throw new ArgumentException("Device map must set the last layer count to -1");

Try / catch

try { deviceMap = model.InferDeviceMapForEachLayer(metadata, entries); } catch (ArgumentException ex) when (ex.Message.Contains("layer count")) { entries[entries.Keys.Last()] = -1; deviceMap = model.InferDeviceMapForEachLayer(metadata, entries); }

Prevention

Always end device map layer counts with -1
Copy device maps only between models of identical architecture/size
Validate layer counts against checkpoint metadata before mapping

When it happens

Trigger: Calling InferDeviceMapForEachLayer (e.g. to prepare llama for multi-device inference) with a device map configuration where no layer entry has count -1, or where declared layer counts do not sum to the model's actual layer count.

Common situations: Hand-written device map configs copied from a differently-sized model checkpoint (e.g. 7B map used for 13B); typo where the last line uses the actual layer count instead of -1.

Understand the failure class

Background: "Invalid value" and "allowed values are" config errors: what your library rejected and how to fix it — this error's family across 41 libraries.

Related errors

AI-assisted analysis of dotnet/machinelearning@7b76e69cf9 (2026-09-11). Data as JSON: /api/errors/969836899dabf946. Report an issue: GitHub.

Appendix: source

Thrown at src/Microsoft.ML.GenAI.Core/Extension/ModuleExtension.cs:245

                {
                    deviceMap[key] = device;
                }
            }
            else
            {
                foreach (var (key, value) in layerSizeMap)
                {
                    deviceMap[key] = device;
                }

                layerSizeMap.Clear();
                break;
            }
        }

        if (layerSizeMap.Count > 0)
        {
            throw new ArgumentException("The layer count is not enough to cover all layers, did you forget to set the last layer count to -1?");
        }

        return deviceMap;
    }

    internal static string Peek(this nn.Module model)
    {
        var sb = new StringBuilder();
        var stateDict = model.state_dict();
        // preview state_dict
        int i = 0;
        foreach (var (key, value) in stateDict.OrderBy(x => x.Key, StringComparer.OrdinalIgnoreCase))
        {
            var str = value.Peek(key);
            sb.AppendLine($"{i}: {str}");
            i++;
        }

View on GitHub (pinned to 7b76e69cf9)