sgl-project/sglang · error · ValueError

num_inference_steps is required for transformer-only mode. P

Error message

num_inference_steps is required for transformer-only mode. Please provide it in CacheDitConfig.

What it means

enable_cache_on_transformer returns early when config.enabled is false; otherwise, in transformer-only mode (no full pipeline object), cache-dit needs the total number of inference steps known up front, so config.num_inference_steps is None raises immediately with this ValueError.

Source

Thrown at python/sglang/multimodal_gen/runtime/cache/cache_dit_integration.py:419

    """Enable cache-dit on a transformer module, by wrapping the module with cache-dit

    This function enables cache-dit acceleration using the BlockAdapterRegister
    for pre-registered models

    Args:
        model_name: Name of the model for logging purposes.
        sp_group: Sequence parallel process group (for Ulysses/Ring).
        tp_group: Tensor parallel process group.
        has_separate_cfg: Whether the run issues separate conditional/unconditional
            passes per step (CFG). Used by custom adapters (ERNIE, Krea-2); a
            mismatch only disables caching, never corrupts output.

    """
    if not config.enabled:
        return transformer

    if config.num_inference_steps is None:
        raise ValueError(
            "num_inference_steps is required for transformer-only mode. "
            "Please provide it in CacheDitConfig."
        )

    # Prefer the standard path (transformer pre-registered in cache-dit). For
    # models absent from the registry, fall back to a manual BlockAdapter (see
    # _build_custom_block_adapter).
    custom_adapter = None
    if not BlockAdapterRegister.is_supported(transformer):
        custom_adapter = _build_custom_block_adapter(
            transformer, has_separate_cfg=has_separate_cfg
        )
        if custom_adapter is None:
            transformer_cls_name = transformer.__class__.__name__
            raise ValueError(
                f"{transformer_cls_name} is not officially supported by cache-dit. "
                "Supported cache-dit DiT families include Flux, QwenImage, HunyuanDiT, "
                "HunyuanVideo, Wan, CogVideoX, Mochi, and others. "

View on GitHub (pinned to 0132848349)

Solutions

1. Set num_inference_steps in CacheDitConfig to match the sampling steps used for requests, e.g. CacheDitConfig(enabled=True, num_inference_steps=50).
2. Plumb the scheduler's/sampler's step count into the config before enabling cache-dit.
3. Disable cache-dit (enabled=False) if you cannot fix the step count.

Example fix

// before
config = CacheDitConfig(enabled=True)
enable_cache_on_transformer(transformer, config)
// after
config = CacheDitConfig(enabled=True, num_inference_steps=50)
enable_cache_on_transformer(transformer, config)

if config.enabled and getattr(config, "num_inference_steps", None) is None:
    raise ValueError("set num_inference_steps before enabling cache-dit")
enable_cache_on_transformer(transformer, config)

def cache_dit_config_valid(config) -> bool:
    return (not config.enabled) or config.num_inference_steps is not None

try:
    enable_cache_on_transformer(transformer, config)
except ValueError as e:
    if "num_inference_steps is required" in str(e):
        config = replace(config, num_inference_steps=sampler_steps)
        retry(enable_cache_on_transformer, transformer, config)
    raise

When it happens

Trigger: Constructing CacheDitConfig(enabled=True) without num_inference_steps and calling enable_cache_on_transformer (typically via _maybe_enable_cache_dit when cache-dit is enabled for a transformer-only runtime).

Common situations: Enabling cache_dit with only request-side overrides and forgetting the server-level step count; switching a deployment from pipeline mode (where steps come from the pipeline) to transformer-only mode; step counts not plumbed from the scheduler into the config.

Related errors

• num_inference_steps is required for dual-transformer mode. P
• LoRA batch_info must provide max_len or seg_lens.
• cache_dit_params must be a dict, got {type(raw).__name__}.
• Unknown cache_dit_params keys: {sorted(unknown)}. Valid keys
• cache_dit_params['secondary'] must be a dict, got {type(seco

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/970d903577d9d96a. Report an issue: GitHub.