{"record":{"id":"edaeeddc62acb789","repo":"sgl-project/sglang","slug":"ltx2durationhead-requires-at-least-one-of-video-to","errorCode":null,"errorMessage":"LTX2DurationHead requires at least one of video_tokens / audio_tokens.","messagePattern":"LTX2DurationHead requires at least one of video_tokens / audio_tokens\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/models/adapter/ltx_2_duration_head.py","lineNumber":92,"sourceCode":"\n        self.attention_pooler = LTX2DurationAttentionPooler(\n            hidden_dim=pooler_hidden_dim,\n            num_queries=arch.num_queries,\n            num_heads=arch.num_pooler_heads,\n        )\n        self.mlp_hidden = nn.Linear(\n            pooler_hidden_dim * arch.num_queries, arch.mlp_hidden_dim\n        )\n        self.mlp_out = nn.Linear(arch.mlp_hidden_dim, 1)\n\n    def forward(\n        self,\n        video_tokens: torch.Tensor | None = None,\n        audio_tokens: torch.Tensor | None = None,\n    ) -> torch.Tensor:\n        \"\"\"Returns predicted duration in seconds, shape `(batch,)`.\"\"\"\n        if video_tokens is None and audio_tokens is None:\n            raise ValueError(\n                \"LTX2DurationHead requires at least one of video_tokens / audio_tokens.\"\n            )\n\n        # The connector output can arrive in a different dtype than the head.\n        head_dtype = self.mlp_out.weight.dtype\n\n        token_groups = []\n        if video_tokens is not None:\n            token_groups.append(\n                self.video_input_proj(video_tokens.to(head_dtype))\n                + self.video_modality_emb\n            )\n        if audio_tokens is not None:\n            token_groups.append(\n                self.audio_input_proj(audio_tokens.to(head_dtype))\n                + self.audio_modality_emb\n            )\n","sourceCodeStart":74,"sourceCodeEnd":110,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/models/adapter/ltx_2_duration_head.py#L74-L110","documentation":"LTX2DurationHead.forward predicts clip duration from connector outputs and requires at least one of video_tokens or audio_tokens. With both None there is no input signal to regress a duration from, so the head refuses rather than producing garbage. Pass whichever modality tokens you have (or both).","triggerScenarios":"Calling `head.forward()` / `head(video_tokens=None, audio_tokens=None)` — e.g. a pipeline step that forwards connector outputs but both fields were dropped, defaulted to None, or an empty batch dict was passed through.","commonSituations":"Wiring a new multimodal pipeline where the audio tower is disabled but video tokens were accidentally not propagated; early prototyping with placeholder None arguments; refactors that renamed connector output keys so token lookup returns None.","solutions":["Pass video_tokens (and/or audio_tokens) from the LTX2 connector output into the duration head call","Check upstream connector forward: verify its returned dict actually contains the token tensors your code reads","Add an assert/logging step in the pipeline that fails earlier with which modality is missing","If running audio-only or video-only configs, make sure the enabled tower's tokens are routed to the head"],"exampleFix":"# before\nseconds = duration_head(video_tokens=batch.get(\"video_tokens\"), audio_tokens=batch.get(\"audio_tokens\"))\n# after\nvideo_tokens = batch.get(\"video_tokens\")\naudio_tokens = batch.get(\"audio_tokens\")\nassert video_tokens is not None or audio_tokens is not None, \"connector produced no tokens\"\nseconds = duration_head(video_tokens=video_tokens, audio_tokens=audio_tokens)","handlingStrategy":"validation","validationCode":"assert video_tokens is not None or audio_tokens is not None, \"LTX2DurationHead needs at least one token tensor\"","typeGuard":"def has_duration_inputs(v: torch.Tensor | None, a: torch.Tensor | None) -> bool:\n    return v is not None or a is not None","tryCatchPattern":"try:\n    seconds = head(video_tokens=v, audio_tokens=a)\nexcept ValueError as e:\n    if \"at least one of video_tokens\" in str(e):\n        raise RuntimeError(f\"connector produced no tokens for batch: {batch_keys}\") from e\n    raise","preventionTips":["Log which modality tensors survive each pipeline stage during bring-up","Default to passing both modalities when available; only omit one intentionally"],"tags":["multimodal","duration-head","argument-validation","ltx-2"],"backgroundTag":"required-argument-missing","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}