sgl-project/sglang · error · ValueError

c2ws_plucker_emb shape must match hidden_states shape, got {

Error message

c2ws_plucker_emb shape must match hidden_states shape, got {tuple(c2ws_plucker_emb.shape)} vs {tuple(hidden_states.shape)}

What it means

When camera Plücker embeddings are injected, LingBotWorld modulates hidden states elementwise, so c2ws_plucker_emb must broadcast exactly — same shape as hidden_states. A shape mismatch (different token count or channel count) means the camera conditioning tensor was built for a different latent layout and is rejected.

Source

Thrown at python/sglang/multimodal_gen/runtime/models/dits/lingbot_world.py:192

    def compute_scale_shift(
        self, c2ws_plucker_emb: torch.Tensor
    ) -> tuple[torch.Tensor, torch.Tensor]:
        c2ws_hidden_states = self.cam_injector(c2ws_plucker_emb)
        c2ws_hidden_states = c2ws_hidden_states + c2ws_plucker_emb
        cam_scale = self.cam_scale_layer(c2ws_hidden_states)
        cam_shift = self.cam_shift_layer(c2ws_hidden_states)
        return cam_scale, cam_shift

    def forward(
        self,
        hidden_states: torch.Tensor,
        c2ws_plucker_emb: torch.Tensor | None,
        scale_shift: tuple[torch.Tensor, torch.Tensor] | None = None,
    ) -> torch.Tensor:
        if c2ws_plucker_emb is None:
            return hidden_states
        if c2ws_plucker_emb.shape != hidden_states.shape:
            raise ValueError(
                "c2ws_plucker_emb shape must match hidden_states shape, "
                f"got {tuple(c2ws_plucker_emb.shape)} vs {tuple(hidden_states.shape)}"
            )
        if scale_shift is None:
            scale_shift = self.compute_scale_shift(c2ws_plucker_emb)
        cam_scale, cam_shift = scale_shift
        return (1.0 + cam_scale) * hidden_states + cam_shift


class LingBotWorldCausalSelfAttention(CausalWanSelfAttention):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        ulysses_world_size = max(get_ulysses_parallel_world_size(), 1)
        if self.num_heads % ulysses_world_size != 0:
            raise ValueError(
                f"num_heads ({self.num_heads}) must be divisible by ulysses_degree ({ulysses_world_size})."
            )
        self.ulysses_num_heads = self.num_heads // ulysses_world_size

View on GitHub (pinned to 0132848349)

Solutions

1. Regenerate c2ws_plucker_emb for the actual latent grid (same tokens, same channels)
2. Check the camera-embedding projector's out_features equals hidden_states.shape[-1] and the token expansion matches
3. If the camera signal is not needed for this layer, pass c2ws_plucker_emb=None (the helper returns hidden_states unchanged)

Example fix

# before
plucker = project_plucker(c2ws)              # (B, T_wrong, D)
h = layer(h, c2ws_plucker_emb=plucker)

# after
plucker = project_plucker(c2ws).expand_to_latents(latents)  # matches (B, S, D)
assert plucker.shape == h.shape
h = layer(h, c2ws_plucker_emb=plucker)

if c2ws_plucker_emb is not None:\n    assert c2ws_plucker_emb.shape == hidden_states.shape, (c2ws_plucker_emb.shape, hidden_states.shape)

def plucker_matches(p: torch.Tensor | None, h: torch.Tensor) -> bool:\n    return p is None or p.shape == h.shape

When it happens

Trigger: Calling the injection helper with c2ws_plucker_emb.shape != hidden_states.shape, e.g. camera embeddings computed for a different number of latent tokens or projected to the wrong width.

Common situations: Changing video resolution/frame count without regenerating Plücker embeddings; camera projector output dim not matching the DiT hidden size after a config edit.

Related errors

• LingBotVideoBlock expects token-level temb6 with shape (B*S,
• The pointers must be multiple of 16 bytes.
• The last dimension ({input.shape[-1]}) x itemsize ({input.dt
• rope_pool_fused expects q/k/v to be 3-D
• rope_pool_fused expects positions/slots to be 1-D

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/aa6dc8834df3aef7. Report an issue: GitHub.