sgl-project/sglang · error · ValueError

num_inference_steps is required for dual-transformer mode. P

Error message

num_inference_steps is required for dual-transformer mode. Please provide it in CacheDitConfig.

What it means

Raised by enable_cache_on_dual_transformer when caching is enabled for a dual-transformer model but primary_config.num_inference_steps is None. Dual-transformer cache-dit schedules cache reuse across a fixed step budget, so the total number of inference steps must be known up front via CacheDitConfig.

Source

Thrown at python/sglang/multimodal_gen/runtime/cache/cache_dit_integration.py:563

    Args:
        primary_config: CacheDitConfig for primary transformer.
        secondary_config: CacheDitConfig for secondary transformer.
        sp_group: Sequence parallel process group (for Ulysses/Ring).
        tp_group: Tensor parallel process group.
    """
    adapter_spec = DUAL_TRANSFORMER_BLOCK_ADAPTER_SPECS.get(model_name)
    if adapter_spec is None:
        raise ValueError(
            f"Dual-transformer cache-dit is only supported for "
            f"{sorted(DUAL_TRANSFORMER_BLOCK_ADAPTER_SPECS)}, got {model_name}."
        )

    if not primary_config.enabled:
        return transformer, transformer_2

    if primary_config.num_inference_steps is None:
        raise ValueError(
            "num_inference_steps is required for dual-transformer mode. "
            "Please provide it in CacheDitConfig."
        )

    # Build DBCacheConfig for primary transformer
    primary_cache_config = DBCacheConfig(
        num_inference_steps=primary_config.num_inference_steps,
        Fn_compute_blocks=primary_config.Fn_compute_blocks,
        Bn_compute_blocks=primary_config.Bn_compute_blocks,
        max_warmup_steps=primary_config.max_warmup_steps,
        residual_diff_threshold=primary_config.residual_diff_threshold,
        max_continuous_cached_steps=primary_config.max_continuous_cached_steps,
        steps_computation_mask=primary_config.steps_computation_mask,
        steps_computation_policy=primary_config.steps_computation_policy,
    )

    # Build DBCacheConfig for secondary transformer
    secondary_cache_config = DBCacheConfig(

View on GitHub (pinned to 0132848349)

Solutions

1. Set num_inference_steps in CacheDitConfig to the sampler's total step count before enabling dual-transformer caching
2. If steps are not known ahead of time, disable caching (enabled=False) for the dual-transformer model
3. Ensure the runtime passes the sampler's num_inference_steps into the config when building it

Example fix

# before
config = CacheDitConfig(enabled=True)  # num_inference_steps=None

# after
config = CacheDitConfig(enabled=True, num_inference_steps=50)  # match your sampler

if is_dual_model and config.cache_dit.enabled and config.cache_dit.num_inference_steps is None:
    config.cache_dit.num_inference_steps = sampler.num_inference_steps
    # or fail fast:
    # raise ValueError("set CacheDitConfig.num_inference_steps for dual-transformer mode")

def dual_config_is_complete(cfg: CacheDitConfig) -> bool:
    return (not cfg.enabled) or cfg.num_inference_steps is not None

try:
    t1, t2 = enable_cache_on_dual_transformer(t1, t2, model_name, cfg, ...)
except ValueError as e:
    if "num_inference_steps is required" in str(e):
        cfg.num_inference_steps = steps; retry once
    else:
        raise

When it happens

Trigger: Constructing CacheDitConfig(enabled=True) without setting num_inference_steps and calling enable_cache_on_dual_transformer (directly or through _maybe_enable_cache_dit) for a supported dual-transformer model.

Common situations: Reusing a single-transformer CacheDitConfig (where num_inference_steps is optional) for a dual-transformer model, or building the config from CLI args that omit the steps setting.

Related errors

• num_inference_steps is required for transformer-only mode. P
• Dual-transformer cache-dit is only supported for {sorted(DUA
• Dual transformers for {model_name} must expose cache-dit blo
• LoRA batch_info must provide max_len or seg_lens.
• num_inference_steps must be positive, got {steps}

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/a7d452a710382c82. Report an issue: GitHub.