huggingface/transformers · error · IndexError

list index out of range

Error message

list index out of range

What it means

`per_layer_config` behaves like a sequence of length `config.num_hidden_layers`. Integer indices outside `[0, num_hidden_layers)` (after the negative-index wrap `layer_idx += len(self)` is applied) raise this standard IndexError, mirroring Python list semantics. It guards the layer-override lookup that follows so `_heterogeneity_spec.per_layer_overrides.get(layer_idx, {})` is only attempted for a real layer.

Source

Thrown at src/transformers/integrations/heterogeneity/configuration_utils.py:253

            layer_overrides = self._config._heterogeneity_spec.per_layer_overrides
            reference_overrides = layer_overrides.get(layer_types.index(layer_idx), {})
            for idx, layer_type in enumerate(layer_types):
                if layer_type == layer_idx and layer_overrides.get(idx, {}) != reference_overrides:
                    raise ValueError(
                        f"Layer type '{layer_idx}' is not homogeneous across layers (layer {idx} differs). "
                        f"Use an integer index to access a specific layer's config."
                    )

            return _get_layer_config(self._config, reference_overrides)

        # Return a list of configs for a slice of layers
        if isinstance(layer_idx, slice):
            return [self[i] for i in range(*layer_idx.indices(len(self)))]

        if layer_idx < 0:
            layer_idx += len(self)
        if layer_idx < 0 or layer_idx >= len(self):
            raise IndexError("list index out of range")

        # Config is actually homogeneous so just return the global config
        if not self._config.is_heterogeneous:
            return self._config

        heterogeneity_spec = self._config._heterogeneity_spec
        return _get_layer_config(
            self._config,
            heterogeneity_spec.per_layer_overrides.get(layer_idx, {}),
        )


def _get_explicit_per_layer_overrides(config: PreTrainedConfig) -> dict[int, dict[str, Any]]:
    heterogeneity_spec = config._heterogeneity_spec
    explicit_per_layer_overrides = {}

    for layer_idx in range(config.num_hidden_layers):
        layer_overrides = copy.deepcopy(heterogeneity_spec.per_layer_overrides.get(layer_idx, {}))

View on GitHub (pinned to a597f97485)

Solutions

1. Derive the bound from the config itself: iterate `range(len(config.per_layer_config))` or `range(config.num_hidden_layers)`.
2. For the last layer use index `-1` or `config.num_hidden_layers - 1`, never `num_hidden_layers`.
3. Validate externally supplied indices before use: `0 <= idx < config.num_hidden_layers`.

Example fix

# before
last = config.per_layer_config[config.num_hidden_layers]  # IndexError

# after
last = config.per_layer_config[-1]

idx = 31
n = config.num_hidden_layers
if not (-n <= idx < n):
    raise IndexError(f"layer {idx} out of range for {n} layers")
layer_cfg = config.per_layer_config[idx]

try:
    layer_cfg = config.per_layer_config[idx]
except IndexError:
    layer_cfg = config.per_layer_config[min(max(idx, 0), config.num_hidden_layers - 1)]  # clamp policy

When it happens

Trigger: Calling `config.per_layer_config[num_hidden_layers]`, `config.per_layer_config[-num_hidden_layers - 1]`, or computing an index from a different model's depth (e.g. hard-coding layer 31 on a 24-layer model).

Common situations: Hard-coded layer indices copied from another checkpoint; loops using `range(0, num_layers + 1)`; off-by-one errors after switching a script to a smaller/larger variant of the same architecture.

Related errors

• out_indices must be valid indices for stage_names {self.stag
• `skip` must be an iterable of strings.
• `skip` must contain only strings.
• `per_layer_config` keys must be integer layer indices in the
• The following layers have the mutually exclusive `sliding_wi

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/12160b86d51c21fb. Report an issue: GitHub.