{"record":{"id":"440e737b27bcebe7","repo":"huggingface/transformers","slug":"per-layer-config-keys-must-be-integer-layer-indi","errorCode":null,"errorMessage":"`per_layer_config` keys must be integer layer indices in the range [0, {num_hidden_layers}); got {invalid_layer_indices}.","messagePattern":"`per_layer_config` keys must be integer layer indices in the range \\[0, (.+?)\\); got (.+?)\\.","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/transformers/integrations/heterogeneity/configuration_utils.py","lineNumber":73,"sourceCode":"        if not all(isinstance(item, str) for item in skip):\n            raise TypeError(\"`skip` must contain only strings.\")\n\n        if skip:\n            normalized[\"skip\"] = sorted(skip)\n\n    return normalized\n\n\ndef _validate_layer_indices(config: PreTrainedConfig, per_layer_overrides: dict[int, dict[str, Any]]) -> None:\n    if not per_layer_overrides:\n        return\n\n    num_hidden_layers = config.num_hidden_layers\n    invalid_layer_indices = [\n        layer_idx for layer_idx in per_layer_overrides if layer_idx < 0 or layer_idx >= num_hidden_layers\n    ]\n    if invalid_layer_indices:\n        raise ValueError(\n            f\"`per_layer_config` keys must be integer layer indices in the range [0, {num_hidden_layers}); \"\n            f\"got {invalid_layer_indices}.\"\n        )\n\n\ndef _validate_sliding_window_and_attention_chunk_size(\n    config: PreTrainedConfig, per_layer_overrides: dict[int, dict[str, Any]]\n) -> None:\n    problematic_indices = []\n    for layer_idx in range(config.num_hidden_layers):\n        layer_overrides = per_layer_overrides.get(layer_idx, {})\n\n        sliding_window = layer_overrides.get(\n            \"sliding_window\", config._getattr_without_heterogeneous_validation(\"sliding_window\", None)\n        )\n        attention_chunk_size = layer_overrides.get(\n            \"attention_chunk_size\",\n            config._getattr_without_heterogeneous_validation(\"attention_chunk_size\", None),","sourceCodeStart":55,"sourceCodeEnd":91,"githubUrl":"https://github.com/huggingface/transformers/blob/a597f974857b3d92939971296bc0deb93d33d780/src/transformers/integrations/heterogeneity/configuration_utils.py#L55-L91","documentation":"_validate_layer_indices checks every key of per_layer_config (the heterogeneity layer-override map) against config.num_hidden_layers: each key must be an integer in [0, num_hidden_layers). Out-of-range indices (negative or >= layer count) are collected and reported in the error, since there is no layer to apply them to.","triggerScenarios":"Setting per_layer_config = {31: {...}} on a model with num_hidden_layers=28, or using -1 as a Python-style 'last layer' index; also mixing in non-integer keys from a JSON round-trip (e.g. strings that compare oddly).","commonSituations":"Porting a per-layer config between model sizes (a 32-layer recipe applied to a 28-layer model); off-by-one layer counts after changing num_hidden_layers; assuming negative indexing works like Python lists.","solutions":["Check model.config.num_hidden_layers and clamp/clip your per_layer_config keys into [0, num_hidden_layers)","Drop overrides for layers that no longer exist when switching model sizes","If keys came out as strings from JSON, convert them with int(k)"],"exampleFix":"# before: model has 28 layers\nconfig.per_layer_config = {0: {...}, 31: {...}}  # ValueError\n\n# after\nn = config.num_hidden_layers\nconfig.per_layer_config = {k: v for k, v in overrides.items() if 0 <= int(k) < n}","handlingStrategy":"validation","validationCode":"n = config.num_hidden_layers\nper_layer_config = {int(k): v for k, v in per_layer_config.items()}\nassert all(0 <= k < n for k in per_layer_config), (\n    f\"keys must be in [0, {n}); got {sorted(k for k in per_layer_config if not 0 <= k < n)}\"\n)\nconfig.per_layer_config = per_layer_config","typeGuard":null,"tryCatchPattern":null,"preventionTips":["When porting per-layer recipes between model sizes, filter keys against num_hidden_layers","Convert JSON string keys to int before assigning per_layer_config","Never use negative indices — they are not Python-style here"],"tags":["heterogeneity","config","index-validation","per-layer"],"backgroundTag":null,"analyzedSha":"a597f974857b3d92939971296bc0deb93d33d780","analyzedAt":"2026-08-14T18:24:08.354Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}