{"record":{"id":"aa6dc8834df3aef7","repo":"sgl-project/sglang","slug":"c2ws-plucker-emb-shape-must-match-hidden-states-sh","errorCode":null,"errorMessage":"c2ws_plucker_emb shape must match hidden_states shape, got {tuple(c2ws_plucker_emb.shape)} vs {tuple(hidden_states.shape)}","messagePattern":"c2ws_plucker_emb shape must match hidden_states shape, got (.+?) vs (.+?)","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/models/dits/lingbot_world.py","lineNumber":192,"sourceCode":"    def compute_scale_shift(\n        self, c2ws_plucker_emb: torch.Tensor\n    ) -> tuple[torch.Tensor, torch.Tensor]:\n        c2ws_hidden_states = self.cam_injector(c2ws_plucker_emb)\n        c2ws_hidden_states = c2ws_hidden_states + c2ws_plucker_emb\n        cam_scale = self.cam_scale_layer(c2ws_hidden_states)\n        cam_shift = self.cam_shift_layer(c2ws_hidden_states)\n        return cam_scale, cam_shift\n\n    def forward(\n        self,\n        hidden_states: torch.Tensor,\n        c2ws_plucker_emb: torch.Tensor | None,\n        scale_shift: tuple[torch.Tensor, torch.Tensor] | None = None,\n    ) -> torch.Tensor:\n        if c2ws_plucker_emb is None:\n            return hidden_states\n        if c2ws_plucker_emb.shape != hidden_states.shape:\n            raise ValueError(\n                \"c2ws_plucker_emb shape must match hidden_states shape, \"\n                f\"got {tuple(c2ws_plucker_emb.shape)} vs {tuple(hidden_states.shape)}\"\n            )\n        if scale_shift is None:\n            scale_shift = self.compute_scale_shift(c2ws_plucker_emb)\n        cam_scale, cam_shift = scale_shift\n        return (1.0 + cam_scale) * hidden_states + cam_shift\n\n\nclass LingBotWorldCausalSelfAttention(CausalWanSelfAttention):\n    def __init__(self, *args, **kwargs):\n        super().__init__(*args, **kwargs)\n        ulysses_world_size = max(get_ulysses_parallel_world_size(), 1)\n        if self.num_heads % ulysses_world_size != 0:\n            raise ValueError(\n                f\"num_heads ({self.num_heads}) must be divisible by ulysses_degree ({ulysses_world_size}).\"\n            )\n        self.ulysses_num_heads = self.num_heads // ulysses_world_size","sourceCodeStart":174,"sourceCodeEnd":210,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/models/dits/lingbot_world.py#L174-L210","documentation":"When camera Plücker embeddings are injected, LingBotWorld modulates hidden states elementwise, so c2ws_plucker_emb must broadcast exactly — same shape as hidden_states. A shape mismatch (different token count or channel count) means the camera conditioning tensor was built for a different latent layout and is rejected.","triggerScenarios":"Calling the injection helper with c2ws_plucker_emb.shape != hidden_states.shape, e.g. camera embeddings computed for a different number of latent tokens or projected to the wrong width.","commonSituations":"Changing video resolution/frame count without regenerating Plücker embeddings; camera projector output dim not matching the DiT hidden size after a config edit.","solutions":["Regenerate c2ws_plucker_emb for the actual latent grid (same tokens, same channels)","Check the camera-embedding projector's out_features equals hidden_states.shape[-1] and the token expansion matches","If the camera signal is not needed for this layer, pass c2ws_plucker_emb=None (the helper returns hidden_states unchanged)"],"exampleFix":"# before\nplucker = project_plucker(c2ws)              # (B, T_wrong, D)\nh = layer(h, c2ws_plucker_emb=plucker)\n\n# after\nplucker = project_plucker(c2ws).expand_to_latents(latents)  # matches (B, S, D)\nassert plucker.shape == h.shape\nh = layer(h, c2ws_plucker_emb=plucker)","handlingStrategy":"validation","validationCode":"if c2ws_plucker_emb is not None:\\n    assert c2ws_plucker_emb.shape == hidden_states.shape, (c2ws_plucker_emb.shape, hidden_states.shape)","typeGuard":"def plucker_matches(p: torch.Tensor | None, h: torch.Tensor) -> bool:\\n    return p is None or p.shape == h.shape","tryCatchPattern":null,"preventionTips":["Regenerate camera embeddings whenever resolution/frame-count changes","Assert projector out_features == hidden size","Pass None when camera control is unused"],"tags":["shape-validation","camera-conditioning","plucker","lingbot"],"backgroundTag":"tensor-shape-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}