huggingface/transformers · error · ValueError

Invalid group type: {}

Error message

Invalid group type: {}

What it means

ValueError raised while building per-group cache allocators from group_layers_by_attn_type(): each layer group's type string must be exactly 'full_attention' or 'sliding_attention'; anything else reaches the else-branch and fails. This indicates an unrecognized layer_type value in the model config (or a naming mismatch with the grouping helper).

Source

Thrown at src/transformers/generation/continuous_batching/cache.py:286

        # Block management data structures
        self.allow_block_sharing = continuous_batching_config.allow_block_sharing
        self.group_cache_managers: list[CacheAllocator] = []
        self.num_full_attention_groups = 0
        self.num_sliding_attention_groups = 0
        self.max_sliding_window_blocks_per_request = 0

        for i, group_type in enumerate(group_types):
            if group_type == "full_attention":
                cm = FullAttentionCacheAllocator(i, self.block_size, allow_block_sharing=self.allow_block_sharing)
                self.num_full_attention_groups += 1
            elif group_type == "sliding_attention":
                cm = SlidingAttentionCacheAllocator(
                    i, self.block_size, config.sliding_window, self.sentinel_index, self.write_trash_index
                )
                self.num_sliding_attention_groups += 1
                self.max_sliding_window_blocks_per_request = cm._max_blocks_per_request
            else:
                raise ValueError(f"Invalid group type: {group_type}")
            self.group_cache_managers.append(cm)

        # We only use prefix sharing if the whole model has only full attention layers and block sharing is allowed
        self.use_prefix_sharing = self.allow_block_sharing and group_types == ["full_attention"]
        self._block_manager = BlockManager(num_blocks, self.block_size, tp_on=tp_size > 1)
        self._total_prefix_length: int = 0  # a counter to measure the impact of prefix sharing, also used in tests

        # For block table support, we lazy init the name of the block table key
        self._block_table_key = None

    def blocks_needed(self, num_requested_blocks: int, allocated_blocks: int) -> int:
        """Returns the number of physical blocks needed to allocate (num_requested_blocks) blocks to a request that
        already has (allocated_blocks) blocks. The number of newly allocated blocks needed is predicted by the
        following rules:
        - for full attention groups: since there is no sliding window for full attention layers, one requested block is
            always equivalent to one newly allocated block for EACH full attention group
        - for sliding window groups: because of the sliding window, the number of blocks allocated to a request is
            capped. Using the number of already (allocated_blocks) we can compute the number of new blocks to actually

View on GitHub (pinned to a597f97485)

Solutions

1. Check config.layer_types (or per-layer layer_type) values and map them to 'full_attention' / 'sliding_attention' before cache construction
2. If the model has a single attention type with no layer_type attribute, remove the attribute so all layers default to one group
3. For unsupported attention flavors, fall back to standard generation instead of continuous batching

Example fix

# before: config.layer_types = ['full', 'sliding', 'full', 'full']
# after
config.layer_types = ['full_attention', 'sliding_attention', 'full_attention', 'full_attention']

VALID = {'full_attention', 'sliding_attention'}
types = getattr(config, 'layer_types', None) or [getattr(config, 'layer_type', 'full_attention')] * config.num_hidden_layers
if any(t not in VALID for t in types):
    raise ValueError(f'unsupported layer_type values: {set(types) - VALID}')

def layer_types_supported(config) -> bool:
    VALID = {'full_attention', 'sliding_attention'}
    types = getattr(config, 'layer_types', None)
    return types is None or all(t in VALID for t in types)

When it happens

Trigger: A model config whose layers carry layer_type values like 'full', 'attention', 'sliding', or a new attention flavor; custom architectures wired into continuous batching with their own layer_type vocabulary; localized-attention variants not yet supported.

Common situations: Supporting new hybrid-attention models; upstream configs that spell types differently than transformers expects.

Related errors

• num_key_value_heads or num_attention_heads could not be foun
• head_dim or (hidden_size and num_attention_heads) could not
• `nbits` for `quanto` backend has to be one of [`2`, `4`] but
• `axis_key` for `quanto` backend has to be one of [`0`, `-1`]
• `axis_value` for `quanto` backend has to be one of [`0`, `-1

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/bfdd1fa9ec4a60df. Report an issue: GitHub.