huggingface/transformers · error · ValueError

Layer type '{layer_idx}' not found in config.layer_types: {l

Error message

Layer type '{layer_idx}' not found in config.layer_types: {layer_types}. Available layer types: {set(layer_types)}

What it means

`per_layer_config` on a heterogeneity-aware config accepts a string key to fetch the shared config of all layers of one type (e.g. "sliding_attention"). The string is matched against `config.layer_types`; if it is not one of the declared layer type names, this ValueError is raised, listing the types that are actually available. It exists to catch typos and layer-type names that the model config simply does not declare.

Source

Thrown at src/transformers/integrations/heterogeneity/configuration_utils.py:225

    return output_config


class _PerLayerConfigView(Sequence["PreTrainedConfig"]):
    def __init__(self, config: PreTrainedConfig) -> None:
        self._config = config

    def __len__(self) -> int:
        return self._config.num_hidden_layers

    def __getitem__(self, layer_idx: int | slice | str) -> PreTrainedConfig | list[PreTrainedConfig]:
        # Return the config for a specific layer type, if the model is homogeneous for that layer type
        if isinstance(layer_idx, str):
            if (layer_types := getattr(self._config, "layer_types", None)) is None:
                raise ValueError(f"Layer type '{layer_idx}' requested, but config.layer_types is not defined. ")

            if layer_idx not in layer_types:
                raise ValueError(
                    f"Layer type '{layer_idx}' not found in config.layer_types: {layer_types}. "
                    f"Available layer types: {set(layer_types)}"
                )

            # Config is actually homogeneous so just return the global config
            if not self._config.is_heterogeneous:
                return self._config

            # Ensure that all layers of the requested type have the same overrides
            layer_overrides = self._config._heterogeneity_spec.per_layer_overrides
            reference_overrides = layer_overrides.get(layer_types.index(layer_idx), {})
            for idx, layer_type in enumerate(layer_types):
                if layer_type == layer_idx and layer_overrides.get(idx, {}) != reference_overrides:
                    raise ValueError(
                        f"Layer type '{layer_idx}' is not homogeneous across layers (layer {idx} differs). "
                        f"Use an integer index to access a specific layer's config."
                    )

View on GitHub (pinned to a597f97485)

Solutions

Print `config.layer_types` and use one of the exact strings listed in the error message.
If you want a specific layer regardless of type, index with an integer: `config.per_layer_config[i]`.
If you want several layers, use a slice: `config.per_layer_config[start:stop]`.
If the model should have that layer type, fix the config (`layer_types=[...]`) before constructing the model.

Example fix

# before
cfg = AutoConfig.from_pretrained(model_id)
layer_cfg = cfg.per_layer_config["attention"]  # ValueError: not in layer_types

# after
print(cfg.layer_types)  # e.g. ['sliding_attention', 'full_attention']
layer_cfg = cfg.per_layer_config["full_attention"]

Defensive patterns

Strategy: validation

Validate before calling

layer_type = "full_attention"
if layer_type not in set(getattr(config, "layer_types", []) or []):
    raise KeyError(f"unknown layer type {layer_type!r}; config declares {config.layer_types}")
layer_cfg = config.per_layer_config[layer_type]

Type guard

def is_valid_layer_type(config, name: str) -> bool:
    return name in set(getattr(config, "layer_types", None) or [])

Try / catch

try:
    layer_cfg = config.per_layer_config[name]
except ValueError as e:
    # message lists available types; re-derive from config.layer_types
    available = set(config.layer_types or [])
    raise KeyError(f"{name!r} not in {available}") from e

Prevention

Never hard-code layer-type strings; always derive them from config.layer_types.
Log config.layer_types once at startup when working with heterogeneous models.
Prefer integer or slice indexing when the layer type is not the point of the access.

When it happens

Trigger: Calling `model.config.per_layer_config["full_attention"]` (or any string index on the PerLayerConfig accessor) when `config.layer_types` is a list like `["sliding_attention", "full_attention"]` that does not contain that string, or when `layer_types` uses different names than the caller assumes (e.g. custom layer type names on a heterogeneous Llama-style model).

Common situations: Working with heterogeneous/mixed-layer models (e.g. models mixing sliding-window and full attention) and guessing the layer-type string instead of reading `config.layer_types`; porting code between models whose `layer_types` vocabulary differs; typos in the layer type name.

Related errors

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/f93d18acd249c544. Report an issue: GitHub.