sgl-project/sglang · error · ValueError

Hunyuan3D reference attention requires a shared cache.

Error message

Hunyuan3D reference attention requires a shared cache.

What it means

In Hunyuan3D Paint's transformer block forward, when a mode is supplied via cross_attention_kwargs the pipeline expects condition_embed_dict — the shared cache that stores per-layer reference embeddings — to be a dict. If mode is set but the cache is missing or not a dict, this ValueError fires because reference attention cannot proceed.

Source

Thrown at python/sglang/multimodal_gen/runtime/models/dits/hunyuan3d_paint.py:101

        scale = scale.unsqueeze(1).repeat(1, num_views).reshape(-1)
        for _ in range(output.ndim - 1):
            scale = scale.unsqueeze(-1)
        return scale

    def forward(
        self,
        hidden_states: torch.Tensor,
        encoder_hidden_states: torch.Tensor,
        attention_mask: torch.Tensor | None = None,
        encoder_attention_mask: torch.Tensor | None = None,
        cross_attention_kwargs: dict[str, Any] | None = None,
    ) -> torch.Tensor:
        options = {} if cross_attention_kwargs is None else cross_attention_kwargs
        num_views = int(options.get("num_in_batch", 1))
        mode = options.get("mode")
        condition_embeddings = options.get("condition_embed_dict")
        if mode is not None and not isinstance(condition_embeddings, dict):
            raise ValueError("Hunyuan3D reference attention requires a shared cache.")

        normalized = self.transformer.norm1(hidden_states)
        hidden_states = hidden_states + self.transformer.attn1(
            normalized, attention_mask=attention_mask
        )

        if mode is not None and "w" in mode:
            condition_embeddings[self.layer_name] = rearrange(
                normalized, "(b n) l c -> b (n l) c", n=num_views
            )

        if mode is not None and "r" in mode and self.use_reference_attention:
            if self.attn_refview is None:
                raise RuntimeError("Reference attention was not initialized.")
            reference = condition_embeddings[self.layer_name]
            reference = reference.unsqueeze(1).repeat(1, num_views, 1, 1)
            reference = rearrange(reference, "b n l c -> (b n) l c")
            reference_output = self.attn_refview(

View on GitHub (pinned to 0132848349)

Solutions

1. Initialize and pass condition_embed_dict as an empty dict in cross_attention_kwargs before the reference/generation passes
2. Run the reference forward pass first so the cache is populated, then the generation pass with mode set
3. Only set 'mode' in cross_attention_kwargs when you actually intend reference/multiview attention

Example fix

# before
out = unet(sample, t, cross_attention_kwargs={"mode": "rw", "num_in_batch": 4})

# after
out = unet(sample, t, cross_attention_kwargs={"mode": "rw", "num_in_batch": 4, "condition_embed_dict": {}})

mode = cross_attention_kwargs.get('mode') if cross_attention_kwargs else None
if mode is not None:
    assert isinstance(cross_attention_kwargs.get('condition_embed_dict'), dict), 'condition_embed_dict required when mode is set'

def has_shared_cache(cross_attention_kwargs) -> bool:
    if not cross_attention_kwargs or cross_attention_kwargs.get('mode') is None:
        return True
    return isinstance(cross_attention_kwargs.get('condition_embed_dict'), dict)

When it happens

Trigger: Calling the paint UNet's block forward with cross_attention_kwargs={'mode': 'rw', ...} but no 'condition_embed_dict' key (or a non-dict value), so there is nowhere to write/read reference view embeddings.

Common situations: Running the paint stage with reference conditioning while a custom pipeline forgot to allocate the shared condition_embed_dict; passing mode for the generation pass without initializing the cache in the reference pass first.

Related errors

• Reference attention was not initialized.
• encoder_hidden_states is required when encoder_key_value is
• Didn't get guidance strength for guidance distilled model.
• Multiview attention was not initialized.
• Hunyuan3D Paint does not use extra UNet conditioning.

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/079238c4ffb28d89. Report an issue: GitHub.