sgl-project/sglang · error · ValueError

Transformer {transformer.__class__.__name__} has no attribut

Error message

Transformer {transformer.__class__.__name__} has no attribute {spec.blocks_attr!r} for cache-dit blocks.

What it means

For transformers not pre-registered with cache-dit, _build_custom_block_adapter looks up a per-class spec in _CUSTOM_BLOCK_ADAPTER_SPECS by class name and reads the blocks via getattr(transformer, spec.blocks_attr, None). If that attribute is None, the class shape doesn't match the spec and adapter construction aborts with this ValueError (raised through enable_cache_on_transformer).

Source

Thrown at python/sglang/multimodal_gen/runtime/cache/cache_dit_integration.py:381

    "MiniMaxH3DiTModel": CustomBlockAdapterSpec(
        blocks_attr="blocks",
        forward_pattern=ForwardPattern.Pattern_3,
    ),
}


def _build_custom_block_adapter(
    transformer: torch.nn.Module,
    has_separate_cfg: bool = False,
) -> Optional[BlockAdapter]:
    """Build a manual BlockAdapter for a model absent from cache-dit's registry,
    or None if the class is unknown."""
    spec = _CUSTOM_BLOCK_ADAPTER_SPECS.get(transformer.__class__.__name__)
    if spec is None:
        return None
    blocks = getattr(transformer, spec.blocks_attr, None)
    if blocks is None:
        raise ValueError(
            f"Transformer {transformer.__class__.__name__} has no attribute "
            f"{spec.blocks_attr!r} for cache-dit blocks."
        )
    return BlockAdapter(
        transformer=transformer,
        blocks=blocks,
        forward_pattern=spec.forward_pattern,
        has_separate_cfg=has_separate_cfg,
    )


def enable_cache_on_transformer(
    transformer: torch.nn.Module,
    config: CacheDitConfig,
    model_name: str = "transformer",
    sp_group: Optional[torch.distributed.ProcessGroup] = None,
    tp_group: Optional[torch.distributed.ProcessGroup] = None,
    has_separate_cfg: bool = False,

View on GitHub (pinned to 0132848349)

Solutions

1. Inspect vars(transformer) to find the actual blocks attribute name and use a transformer whose layout matches the spec.
2. Update the _CUSTOM_BLOCK_ADAPTER_SPECS entry for that class to the correct blocks_attr.
3. Pre-register the transformer with cache-dit so the standard path is used and the custom adapter is not needed.

Example fix

// before
# spec expects transformer.blocks but model defines transformer.transformer_blocks
enable_cache_on_transformer(model, config)
// after
# update spec's blocks_attr (or model revision) so getattr finds blocks
enable_cache_on_transformer(model, config)

spec = _CUSTOM_BLOCK_ADAPTER_SPECS.get(type(transformer).__name__)
if spec is not None and getattr(transformer, spec.blocks_attr, None) is None:
    raise RuntimeError(f"{type(transformer).__name__} missing {spec.blocks_attr}; update spec")
enable_cache_on_transformer(transformer, config)

def transformer_supports_adapter(transformer) -> bool:
    spec = _CUSTOM_BLOCK_ADAPTER_SPECS.get(type(transformer).__name__)
    return spec is None or getattr(transformer, spec.blocks_attr, None) is not None

try:
    enable_cache_on_transformer(transformer, config)
except ValueError as e:
    if "for cache-dit blocks" in str(e):
        config = replace(config, enabled=False)  # serve without cache-dit
    raise

When it happens

Trigger: Enabling cache-dit on a transformer whose class name has a spec in _CUSTOM_BLOCK_ADAPTER_SPECS but whose instance lacks the expected blocks_attr — e.g. the blocks attribute was renamed in a newer model revision, or a wrapper module hides it.

Common situations: Model code updates renaming transformer.blocks; loading a custom/sharded checkpoint that reuses a known class name with different internals; passing a partially-initialized transformer before attributes are assigned.

Related errors

• {transformer_cls_name} is not officially supported by cache-
• cache_dit_params must be a dict, got {type(raw).__name__}.
• Unknown cache_dit_params keys: {sorted(unknown)}. Valid keys
• cache_dit_params['secondary'] must be a dict, got {type(seco
• Unknown cache_dit_params['secondary'] keys: {sorted(unknown)

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/b07f9e202e6d5fc1. Report an issue: GitHub.