sgl-project/sglang · error · RuntimeError

Reference attention was not initialized.

Error message

Reference attention was not initialized.

What it means

When mode contains 'r' (reference attention) and use_reference_attention is enabled, the block requires a self.attn_refview module. If it is None — i.e. reference attention was never initialized — this RuntimeError is raised before attempting the reference attention call.

Source

Thrown at python/sglang/multimodal_gen/runtime/models/dits/hunyuan3d_paint.py:115

        num_views = int(options.get("num_in_batch", 1))
        mode = options.get("mode")
        condition_embeddings = options.get("condition_embed_dict")
        if mode is not None and not isinstance(condition_embeddings, dict):
            raise ValueError("Hunyuan3D reference attention requires a shared cache.")

        normalized = self.transformer.norm1(hidden_states)
        hidden_states = hidden_states + self.transformer.attn1(
            normalized, attention_mask=attention_mask
        )

        if mode is not None and "w" in mode:
            condition_embeddings[self.layer_name] = rearrange(
                normalized, "(b n) l c -> b (n l) c", n=num_views
            )

        if mode is not None and "r" in mode and self.use_reference_attention:
            if self.attn_refview is None:
                raise RuntimeError("Reference attention was not initialized.")
            reference = condition_embeddings[self.layer_name]
            reference = reference.unsqueeze(1).repeat(1, num_views, 1, 1)
            reference = rearrange(reference, "b n l c -> (b n) l c")
            reference_output = self.attn_refview(
                normalized, encoder_hidden_states=reference
            )
            reference_scale = self._broadcast_scale(
                1.0 if self.is_turbo else options.get("ref_scale", 1.0),
                reference_output,
                num_views,
            )
            hidden_states = hidden_states + reference_scale * reference_output

        if num_views > 1 and self.use_multiview_attention:
            if self.attn_multiview is None:
                raise RuntimeError("Multiview attention was not initialized.")
            multiview = rearrange(normalized, "(b n) l c -> b (n l) c", n=num_views)
            position_masks = options.get("position_attn_mask")

View on GitHub (pinned to 0132848349)

Solutions

1. Construct/replace the transformer blocks with use_reference_attention=True so attn_refview is initialized
2. Verify the layer matches one that received reference attention during _replace_transformer_blocks
3. Don't include 'r' in mode for blocks without reference attention

Example fix

# before
_replace_transformer_blocks(unet, use_reference_attention=False)
out = block(x, cross_attention_kwargs={"mode": "rw", ...})

# after
_replace_transformer_blocks(unet, use_reference_attention=True)
out = block(x, cross_attention_kwargs={"mode": "rw", "condition_embed_dict": cache})

if 'r' in mode and getattr(block, 'use_reference_attention', False):
    assert block.attn_refview is not None, 'attn_refview not initialized'

def block_has_reference_attn(block) -> bool:
    return getattr(block, 'use_reference_attention', False) and block.attn_refview is not None

try:
    out = block(x, cross_attention_kwargs=opts)
except RuntimeError as e:
    if 'Reference attention was not initialized' in str(e):
        opts = {**opts, 'mode': opts['mode'].replace('r', '')}
        out = block(x, cross_attention_kwargs=opts)
    else:
        raise

When it happens

Trigger: Calling forward with mode='r...' on a block where attn_refview was never created — e.g. use_reference_attention was False (or the wrong layer) at construction, or blocks were replaced without reference attention for this layer_name.

Common situations: Mixing a checkpoint/pipeline that enables reference attention at runtime with model blocks built with use_reference_attention=False; layer names in the cache not matching blocks that own attn_refview.

Related errors

• Hunyuan3D reference attention requires a shared cache.
• Multiview attention was not initialized.
• encoder_hidden_states is required when encoder_key_value is
• Didn't get guidance strength for guidance distilled model.
• Hunyuan3D Paint does not use extra UNet conditioning.

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/b56cb0dbdcc4bb52. Report an issue: GitHub.