huggingface/transformers · error · ValueError

`num_hidden_layers` ({num_hidden_layers}) must be equal to t

Error message

`num_hidden_layers` ({num_hidden_layers}) must be equal to the number of layer types ({len(layer_types)})

What it means

ValueError from layer_type_validation (deprecated free function): the length of layer_types must equal num_hidden_layers exactly, because every hidden layer needs one type entry. Passing a mismatched pair fails even when all entries are individually valid.

Source

Thrown at src/transformers/configuration_utils.py:1491

    PreTrainedConfig.push_to_hub.__doc__ = PreTrainedConfig.push_to_hub.__doc__.format(
        object="config", object_class="AutoConfig", object_files="configuration file"
    )


# The alias is only here for BC - we did not have the correct CamelCasing before
PretrainedConfig = PreTrainedConfig


def layer_type_validation(layer_types: list[str], num_hidden_layers: int | None = None, attention: bool = True):
    logger.warning(
        "`layer_type_validation` is deprecated and will be removed in v5.20. "
        "Use `PreTrainedConfig.validate_layer_type` instead"
    )

    if not all(layer_type in ALLOWED_LAYER_TYPES for layer_type in layer_types):
        raise ValueError(f"The `layer_types` entries must be in {ALLOWED_LAYER_TYPES}")
    if num_hidden_layers is not None and num_hidden_layers != len(layer_types):
        raise ValueError(
            f"`num_hidden_layers` ({num_hidden_layers}) must be equal to the number of layer types "
            f"({len(layer_types)})"
        )

View on GitHub (pinned to a597f97485)

Solutions

1. Regenerate layer_types to match depth: config.layer_types = ["dense"] * config.num_hidden_layers (or the intended pattern of that length).
2. When changing num_hidden_layers on an existing config, update layer_types in the same edit.
3. Pass num_hidden_layers=None if you only want entry validation and will validate the count elsewhere.

Example fix

// before
layer_type_validation(["dense"] * 3, num_hidden_layers=4)  # ValueError

// after
layer_type_validation(["dense"] * 4, num_hidden_layers=4)

assert num_hidden_layers is None or num_hidden_layers == len(layer_types), (
    f"num_hidden_layers={num_hidden_layers} != len(layer_types)={len(layer_types)}")

def layer_types_match_depth(layer_types: list[str], num_hidden_layers: int | None) -> bool:
    return num_hidden_layers is None or num_hidden_layers == len(layer_types)

When it happens

Trigger: layer_type_validation(["dense", "linear"], num_hidden_layers=32), or configs where num_hidden_layers was updated (e.g. reduced for a smaller variant) but layer_types was not regenerated.

Common situations: Scaling configs up/down programmatically; layer-type lists built by repetition with the wrong factor; MTP or distilled variants that changed depth.

Related errors

• The `layer_types` entries must be in {ALLOWED_LAYER_TYPES}
• The `{layer_types}` entries must be in {allowed_types} but g
• `num_hidden_layers` ({self.num_hidden_layers}) must be equal
• Layer type '{layer_idx}' not found in config.layer_types: {l
• out_indices must be a list, got {type(self._out_indices)}

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/874e32f7ad3bc61e. Report an issue: GitHub.