sgl-project/sglang · error · ValueError

Didn't get guidance strength for guidance distilled model.

Error message

Didn't get guidance strength for guidance distilled model.

What it means

The Hunyuan3D transformer was built with guidance_embed=True (guidance-distilled), so every forward pass needs a guidance value to embed and add to the timestep vector. forward raises this ValueError when kwargs contains no 'guidance' key (or it is None).

Source

Thrown at python/sglang/multimodal_gen/runtime/models/dits/hunyuan3d.py:589

        contexts,
        **kwargs,
    ) -> torch.Tensor:
        """Forward pass for denoising."""

        cond = contexts["main"]

        latent = self.latent_in(x)

        t_emb = _flux_timestep_embedding(t, 256, self.time_factor).to(
            dtype=latent.dtype
        )

        vec = self.time_in(t_emb)

        if self.guidance_embed:
            guidance = kwargs.get("guidance", None)
            if guidance is None:
                raise ValueError(
                    "Didn't get guidance strength for guidance distilled model."
                )
            vec = vec + self.guidance_in(
                _flux_timestep_embedding(guidance, 256, self.time_factor)
            )

        cond = self.cond_in(cond)

        pe = None

        # Double blocks
        for i, block in enumerate(self.double_blocks):
            latent, cond = block(img=latent, txt=cond, vec=vec, pe=pe)
        latent = torch.cat((cond, latent), 1)

        # Single blocks
        for i, block in enumerate(self.single_blocks):
            latent = block(latent, vec=vec, pe=pe)

View on GitHub (pinned to 0132848349)

Solutions

1. Pass guidance=... in the model kwargs each step (typical distilled value ~3.5 for Hunyuan3D)
2. If you want classifier-free guidance instead, load/use a non-distilled config where guidance_embed=False
3. Double-check your pipeline builds kwargs with the 'guidance' key for distilled checkpoints

Example fix

# before
out = model(hidden_states, timestep=t, context=ctx)

# after
out = model(hidden_states, timestep=t, context=ctx, guidance=torch.tensor([3.5]))

if model.guidance_embed:
    assert kwargs.get('guidance') is not None, 'distilled model requires kwargs["guidance"]'

def needs_guidance(model) -> bool:
    return getattr(model, 'guidance_embed', False)

try:
    out = model(h, t, **kwargs)
except ValueError as e:
    if 'guidance strength' in str(e) and model.guidance_embed:
        kwargs['guidance'] = torch.tensor([3.5], device=h.device)
        out = model(h, t, **kwargs)
    else:
        raise

When it happens

Trigger: Calling model forward on a guidance-distilled Hunyuan3D checkpoint without kwargs['guidance'], e.g. running a CFG-style sampling loop that assumes no guidance embedding is needed.

Common situations: Using a distillation-capable checkpoint with a sampler written for the non-distilled model; guidance key dropped when building a kwargs dict dynamically.

Related errors

• encoder_hidden_states is required when encoder_key_value is
• vis_freqs_cis is required for fused QK-Norm + RoPE kernel
• Hunyuan3D reference attention requires a shared cache.
• Reference attention was not initialized.
• Multiview attention was not initialized.

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/b0b791c4e26099d7. Report an issue: GitHub.