huggingface/transformers · error · ValueError

Layer type '{layer_idx}' requested, but config.layer_types i

Error message

Layer type '{layer_idx}' requested, but config.layer_types is not defined. 

What it means

_PerLayerConfigView.__getitem__ supports indexing by layer-type name (e.g. config.per_layer_config["full"] or ["linear_attention"]) for models whose config declares layer_types. That string-indexing path requires config.layer_types to exist; if the attribute is absent (the model never declared layer types), the view cannot map a name to layers and raises ValueError. Note the message has a trailing space and mentions the requested name, not the available set.

Source

Thrown at src/transformers/integrations/heterogeneity/configuration_utils.py:222

        if attr == "skip":
            continue
        setattr(output_config, attr, value)

    return output_config


class _PerLayerConfigView(Sequence["PreTrainedConfig"]):
    def __init__(self, config: PreTrainedConfig) -> None:
        self._config = config

    def __len__(self) -> int:
        return self._config.num_hidden_layers

    def __getitem__(self, layer_idx: int | slice | str) -> PreTrainedConfig | list[PreTrainedConfig]:
        # Return the config for a specific layer type, if the model is homogeneous for that layer type
        if isinstance(layer_idx, str):
            if (layer_types := getattr(self._config, "layer_types", None)) is None:
                raise ValueError(f"Layer type '{layer_idx}' requested, but config.layer_types is not defined. ")

            if layer_idx not in layer_types:
                raise ValueError(
                    f"Layer type '{layer_idx}' not found in config.layer_types: {layer_types}. "
                    f"Available layer types: {set(layer_types)}"
                )

            # Config is actually homogeneous so just return the global config
            if not self._config.is_heterogeneous:
                return self._config

            # Ensure that all layers of the requested type have the same overrides
            layer_overrides = self._config._heterogeneity_spec.per_layer_overrides
            reference_overrides = layer_overrides.get(layer_types.index(layer_idx), {})
            for idx, layer_type in enumerate(layer_types):
                if layer_type == layer_idx and layer_overrides.get(idx, {}) != reference_overrides:
                    raise ValueError(
                        f"Layer type '{layer_idx}' is not homogeneous across layers (layer {idx} differs). "

View on GitHub (pinned to a597f97485)

Solutions

1. Index by integer layer index instead of type name when layer_types is undefined
2. If the model should have layer types, add config.layer_types = ["full", "sliding", ...] with one entry per layer
3. Guard string indexing with hasattr(config, "layer_types")

Example fix

# before
cfg = AutoConfig.from_pretrained(model_id)  # no layer_types
cfg.per_layer_config["sliding"]  # ValueError

# after
if getattr(cfg, "layer_types", None) is not None:
    view = cfg.per_layer_config["sliding"]
else:
    view = cfg.per_layer_config[0]

if getattr(config, "layer_types", None) is None:
    layer_cfg = config.per_layer_config[0]  # integer index is always safe
else:
    layer_cfg = config.per_layer_config["sliding"]

def supports_layer_type_indexing(config) -> bool:
    """True only when config.layer_types is declared (required for string indexing)."""
    return getattr(config, "layer_types", None) is not None

When it happens

Trigger: Calling config.per_layer_config["some_type"] on a model config that has no layer_types attribute — e.g. a homogeneous model, or a heterogeneous config built only with numeric indices.

Common situations: Generic code that assumes every model exposes layer_types (they only appeared with hybrid/heterogeneous architectures like Gemma-3, xLSTM-style models); accessing per-layer views on older checkpoints whose configs predate the field.

Related errors

• Layer type '{layer_idx}' not found in config.layer_types: {l
• The `{layer_types}` entries must be in {allowed_types} but g
• `num_hidden_layers` ({self.num_hidden_layers}) must be equal
• Calling `get_mtp_config` on a config containing `layer_types
• The `layer_types` entries must be in {ALLOWED_LAYER_TYPES}

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/c889a28d98e7cbcd. Report an issue: GitHub.