vllm-project/vllm · error · ValueError

The model is an hybrid without a layers_block_type or an att

Error message

The model is an hybrid without a layers_block_type or an attn_type_list, or a layer_types in the hf_config, cannot determine the num of {block_type} layers

What it means

Hybrid models (mixing full attention with linear/slab attention) need the per-layer type map to count layers of a given block type. get_num_layers_by_block_type raises when none of layers_block_type, attn_type_list, or layer_types is present in the HF config, so the count cannot be determined.

Source

Thrown at vllm/config/model.py:1598

            layer_types_value = getattr(self.hf_text_config, "layer_types", None)
            if layer_types_value is not None:
                if block_type == "attention":
                    return sum(
                        t == "full_attention" for t in layer_types_value[start:end]
                    )
                elif block_type == "linear_attention":
                    return sum(
                        t == "linear_attention" for t in layer_types_value[start:end]
                    )
                else:
                    return sum(t == block_type for t in layer_types_value[start:end])

            if (
                layers_block_type_value is None
                and attn_type_list is None
                and layer_types_value is None
            ):
                raise ValueError(
                    "The model is an hybrid without a layers_block_type or an "
                    "attn_type_list, or a layer_types in the hf_config, "
                    f"cannot determine the num of {block_type} layers"
                )
            raise AssertionError(f"Unsupported block type: {block_type}")

    def get_mamba_chunk_size(self) -> int:
        """
        Returns the mamba chunk size if it exists
        """
        # used by e.g. Bamba, FalconH1, Granite
        chunk_size = getattr(self.hf_text_config, "mamba_chunk_size", None)
        if chunk_size is None:
            # used by e.g. Mamba2, NemotronH, Zamba
            chunk_size = getattr(self.hf_text_config, "chunk_size", None)

        # Since Mamba1 does not have a chunk notion
        # we use a default chunk size of 2048.

View on GitHub (pinned to c794754062)

Solutions

Inspect the checkpoint's config.json and add the correct layer-type list (layer_types / attn_type_list / layers_block_type) matching the released model.
Re-download or re-convert the checkpoint with the same tooling/version the model author used, so hybrid fields survive.
Update vLLM/HF transformers to a version that knows this architecture's config field name.

Example fix

// before: config.json
{"architectures": ["MyHybridForCausalLM"], "num_hidden_layers": 24}
// after
{"architectures": ["MyHybridForCausalLM"], "num_hidden_layers": 24,
 "layer_types": ["attention", "linear_attention"]}

Defensive patterns

Strategy: type-guard

Validate before calling

def has_layer_types(hf_text_config) -> bool:
    return any(
        getattr(hf_text_config, f, None) is not None
        for f in ('layers_block_type', 'attn_type_list', 'layer_types')
    )
# gate hybrid-model paths on this before counting block types

Type guard

def is_valid_hybrid_config(cfg) -> bool:
    return getattr(cfg, 'layer_types', None) is not None or \
           getattr(cfg, 'attn_type_list', None) is not None or \
           getattr(cfg, 'layers_block_type', None) is not None

Try / catch

except ValueError as e:
    if 'cannot determine the num of' in str(e):
        fail with a clear message telling the user to fix the checkpoint's config.json layer-types field

Prevention

Validate hybrid checkpoint configs (layer_types present, length == num_hidden_layers) before serving.
Keep conversion tooling versions aligned with the model author's to preserve hybrid config fields.
Add a checkpoint smoke test that loads ModelConfig in CI for every supported hybrid architecture.

When it happens

Trigger: Calling get_num_layers_by_block_type on a model flagged as hybrid (or a caller like a sampler/scheduler needing the count) whose hf_text_config lacks all three layer-type fields.

Common situations: Loading a hybrid architecture (e.g. Zamba-, Bamba-, FalconH1-style or custom SSM hybrids) checkpoint whose config.json omits or renames the layer-type list; converting checkpoints with tooling that drops extra config fields; upstream HF config schema changes between versions.

Related errors

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/d8b9f40a41122ce8. Report an issue: GitHub.