{"record":{"id":"648cfa5761d2a4ba","repo":"huggingface/transformers","slug":"layer-type-layer-idx-is-not-homogeneous-across","errorCode":null,"errorMessage":"Layer type '{layer_idx}' is not homogeneous across layers (layer {idx} differs). Use an integer index to access a specific layer's config.","messagePattern":"Layer type '(.+?)' is not homogeneous across layers \\(layer (.+?) differs\\)\\. Use an integer index to access a specific layer's config\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/transformers/integrations/heterogeneity/configuration_utils.py","lineNumber":239,"sourceCode":"            if (layer_types := getattr(self._config, \"layer_types\", None)) is None:\n                raise ValueError(f\"Layer type '{layer_idx}' requested, but config.layer_types is not defined. \")\n\n            if layer_idx not in layer_types:\n                raise ValueError(\n                    f\"Layer type '{layer_idx}' not found in config.layer_types: {layer_types}. \"\n                    f\"Available layer types: {set(layer_types)}\"\n                )\n\n            # Config is actually homogeneous so just return the global config\n            if not self._config.is_heterogeneous:\n                return self._config\n\n            # Ensure that all layers of the requested type have the same overrides\n            layer_overrides = self._config._heterogeneity_spec.per_layer_overrides\n            reference_overrides = layer_overrides.get(layer_types.index(layer_idx), {})\n            for idx, layer_type in enumerate(layer_types):\n                if layer_type == layer_idx and layer_overrides.get(idx, {}) != reference_overrides:\n                    raise ValueError(\n                        f\"Layer type '{layer_idx}' is not homogeneous across layers (layer {idx} differs). \"\n                        f\"Use an integer index to access a specific layer's config.\"\n                    )\n\n            return _get_layer_config(self._config, reference_overrides)\n\n        # Return a list of configs for a slice of layers\n        if isinstance(layer_idx, slice):\n            return [self[i] for i in range(*layer_idx.indices(len(self)))]\n\n        if layer_idx < 0:\n            layer_idx += len(self)\n        if layer_idx < 0 or layer_idx >= len(self):\n            raise IndexError(\"list index out of range\")\n\n        # Config is actually homogeneous so just return the global config\n        if not self._config.is_heterogeneous:\n            return self._config","sourceCodeStart":221,"sourceCodeEnd":257,"githubUrl":"https://github.com/huggingface/transformers/blob/a597f974857b3d92939971296bc0deb93d33d780/src/transformers/integrations/heterogeneity/configuration_utils.py#L221-L257","documentation":"When `per_layer_config` is indexed by a layer-type string on a heterogeneous config, transformers verifies that every layer of that type carries identical per-layer overrides, so a single shared config can be returned. If any layer of the requested type has different overrides (e.g. a different head dim or rope setting injected per layer), the value is ambiguous and this ValueError is raised, directing you to integer indexing.","triggerScenarios":"Calling `config.per_layer_config[\"sliding_attention\"]` on a config where `is_heterogeneous` is True and `config._heterogeneity_spec.per_layer_overrides` maps two layers both typed \"sliding_attention\" to different override dicts (e.g. layer 3 has {\"num_key_value_heads\": 4} but layer 9 has {}).","commonSituations":"Custom heterogeneous checkpoints built with per-layer overrides that only partially cover the layers of one type; programmatically generating `per_layer_overrides` with off-by-one or conditional logic so layers of the same type diverge; assuming a layer type implies uniform layer config.","solutions":["Index the specific layer you care about with an integer: `config.per_layer_config[3]`.","Iterate all layers and filter by type yourself: `[c for i, c in enumerate(config.per_layer_config) if config.layer_types[i] == \"sliding_attention\"]`.","If the layers of that type are supposed to be identical, fix the construction of `per_layer_overrides` so every layer of the type gets the same override dict."],"exampleFix":"# before\nshared = config.per_layer_config[\"sliding_attention\"]  # layers of this type differ\n\n# after\nlayer_cfgs = [\n    config.per_layer_config[i]\n    for i, t in enumerate(config.layer_types)\n    if t == \"sliding_attention\"\n]","handlingStrategy":"validation","validationCode":"def layer_type_is_homogeneous(config, layer_type: str) -> bool:\n    types = config.layer_types\n    ov = config._heterogeneity_spec.per_layer_overrides\n    ref = ov.get(types.index(layer_type), {})\n    return all(ov.get(i, {}) == ref for i, t in enumerate(types) if t == layer_type)\n\nif layer_type_is_homogeneous(config, \"sliding_attention\"):\n    shared = config.per_layer_config[\"sliding_attention\"]","typeGuard":"def can_use_string_index(config, layer_type: str) -> bool:\n    types = getattr(config, \"layer_types\", None)\n    if not types or layer_type not in types:\n        return False\n    return not config.is_heterogeneous or layer_type_is_homogeneous(config, layer_type)","tryCatchPattern":"try:\n    shared = config.per_layer_config[layer_type]\nexcept ValueError:\n    shared = config.per_layer_config[config.layer_types.index(layer_type)]  # first layer of that type","preventionTips":["Treat string indexing as an optimization, not a guarantee; keep an integer-index fallback.","When building per-layer overrides programmatically, assert all layers of a type share one override dict.","Write helpers that iterate layers by type instead of assuming uniformity."],"tags":["config","heterogeneity","per-layer-overrides","validation"],"backgroundTag":null,"analyzedSha":"a597f974857b3d92939971296bc0deb93d33d780","analyzedAt":"2026-08-14T18:24:08.354Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}