sgl-project/sglang · error · NotImplementedError

teacache is not supported yet for HunyuanVideo

Error message

teacache is not supported yet for HunyuanVideo

What it means

HunyuanVideo's should_skip_forward_for_cached_states returns False when TeaCache is disabled, but if forward_batch.enable_teacache is True it raises NotImplementedError — TeaCache acceleration has simply not been implemented for this model yet (the code after the raise is dead/placeholder).

Source

Thrown at python/sglang/multimodal_gen/runtime/models/dits/hunyuanvideo.py:959

        return img

    def maybe_cache_states(
        self, hidden_states: torch.Tensor, original_hidden_states: torch.Tensor
    ) -> None:
        self.previous_residual = hidden_states - original_hidden_states

    def should_skip_forward_for_cached_states(self, **kwargs) -> bool:
        forward_context = get_forward_context()
        forward_batch = forward_context.forward_batch
        if forward_batch is None:
            return False
        current_timestep = forward_context.current_timestep
        enable_teacache = forward_batch.enable_teacache

        if not enable_teacache:
            return False
        raise NotImplementedError("teacache is not supported yet for HunyuanVideo")

        teacache_params = forward_batch.teacache_params
        assert teacache_params is not None, "teacache_params is not initialized"
        assert isinstance(
            teacache_params, TeaCacheParams
        ), "teacache_params is not a TeaCacheParams"
        num_inference_steps = forward_batch.num_inference_steps
        teache_thresh = teacache_params.teacache_thresh

        coefficients = teacache_params.coefficients

        if current_timestep == 0:
            self.cnt = 0

        inp = kwargs["img"].clone()
        vec_ = kwargs["vec"].clone()
        # convert to DTensor
        vec_ = torch.distributed.tensor.DTensor.from_local(

View on GitHub (pinned to 0132848349)

Solutions

1. Disable teacache for HunyuanVideo runs (remove --enable-teacache / set enable_teacache=False)
2. Keep model-specific flag sets per model in your launch scripts instead of one shared flag list
3. Watch releases for TeaCache support landing for HunyuanVideo before re-enabling

Example fix

# before
forward_batch.enable_teacache = True  # or --enable-teacache on server start

# after
forward_batch.enable_teacache = False  # remove the teacache flag for HunyuanVideo

if forward_batch.enable_teacache and model_config.architecture == 'HunyuanVideo':
    forward_batch.enable_teacache = False  # not supported; run without acceleration
    logger.warning('teacache disabled: unsupported for HunyuanVideo')

def model_supports_teacache(model_name: str) -> bool:
    return model_name.lower() != 'hunyuanvideo'

try:
    out = model.forward(hidden_states, forward_batch)
except NotImplementedError as e:
    if 'teacache' in str(e):
        forward_batch.enable_teacache = False
        out = model.forward(hidden_states, forward_batch)
    else:
        raise

When it happens

Trigger: Running HunyuanVideo inference with the teacache flag enabled (forward_batch.enable_teacache=True), e.g. passing --enable-teacache or the equivalent server/runtime option.

Common situations: Copying launch flags from other DiT models that do support TeaCache; enabling teacache in a shared serving config that routes multiple video models.

Related errors

• Block sparsity + paged KV not supported on SM100
• Block sparsity + sheared bias is not supported on SM90
• Custom user-provided score_mod is not supported on SM8x arch
• Validate failed: S({S}) must be divisible by F({F}).
• Only neox-style RoPE is supported.

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/93a8b3354a5de951. Report an issue: GitHub.