{"record":{"id":"acd0be3088673f16","repo":"Comfy-Org/ComfyUI","slug":"durationhead-requires-at-least-one-of-video-tokens","errorCode":null,"errorMessage":"DurationHead requires at least one of video_tokens / audio_tokens","messagePattern":"DurationHead requires at least one of video_tokens / audio_tokens","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"comfy/ldm/lightricks/duration_head.py","lineNumber":56,"sourceCode":"        self.video_input_proj = nn.Linear(video_cross_attention_dim, pooler_hidden_dim)\n        self.video_modality_emb = nn.Parameter(torch.empty(pooler_hidden_dim))\n        self.audio_input_proj = nn.Linear(audio_cross_attention_dim, pooler_hidden_dim)\n        self.audio_modality_emb = nn.Parameter(torch.empty(pooler_hidden_dim))\n        self.attention_pooler = AttentionPooler(\n            hidden_dim=pooler_hidden_dim, num_queries=num_queries, num_heads=num_pooler_heads)\n        self.mlp_hidden = nn.Linear(pooler_hidden_dim * num_queries, mlp_hidden)\n        self.mlp_out = nn.Linear(mlp_hidden, 1)\n\n    def forward(self, video_tokens=None, audio_tokens=None):\n        \"\"\"``video_tokens``: (B, T_v, 4096), ``audio_tokens``: (B, T_a, 2048);\n        at least one required. Returns duration in seconds, shape (B,).\"\"\"\n        token_groups = []\n        if video_tokens is not None:\n            token_groups.append(self.video_input_proj(video_tokens) + self.video_modality_emb)\n        if audio_tokens is not None:\n            token_groups.append(self.audio_input_proj(audio_tokens) + self.audio_modality_emb)\n        if not token_groups:\n            raise ValueError(\"DurationHead requires at least one of video_tokens / audio_tokens\")\n        pooled = self.attention_pooler(torch.cat(token_groups, dim=1))\n        pooled = pooled.reshape(pooled.shape[0], -1)\n        hidden = F.gelu(self.mlp_hidden(pooled), approximate=\"tanh\")\n        return self.mlp_out(hidden).squeeze(-1).exp()\n\n\ndef normalize_state_dict(sd):\n    for prefix in (\"model.diffusion_model.duration_head.\", \"duration_head.\"):\n        stripped = {k[len(prefix):]: v for k, v in sd.items() if k.startswith(prefix)}\n        if stripped:\n            return stripped\n    return sd\n\n\ndef seconds_to_num_frames(seconds, frame_rate, min_seconds, max_seconds, time_scale=8):\n    \"\"\"Convert seconds to a frame count clamped to ``[min_seconds, max_seconds]``\n    and snapped (floor) to the VAE's ``8k + 1`` causal temporal grid; snapping\n    that undershoots the minimum bumps up to the next grid point instead.\"\"\"","sourceCodeStart":38,"sourceCodeEnd":74,"githubUrl":"https://github.com/Comfy-Org/ComfyUI/blob/1c6d8d45b3693bfbb32385b410d813a7fd6be216/comfy/ldm/lightricks/duration_head.py#L38-L74","documentation":"LTX 2.4's DurationHead predicts shot duration from connector tokens; its forward needs at least one of video_tokens (B, T_v, 4096) or audio_tokens (B, T_a, 2048) to pool over. Calling it with both None leaves the attention pooler with an empty sequence, so it raises immediately.","triggerScenarios":"duration_head.forward() / model call with both connector outputs omitted, e.g. running a text-only path through a node that always instantiates the head, or a script that passes tokens=None when a modality is disabled.","commonSituations":"Disabling both audio and video connectors while keeping the duration head enabled, or refactoring a pipeline where tokens were previously positional and got shifted to None.","solutions":["Pass at least one modality's connector output (usually the caption/video tokens)","Skip the duration-head call entirely when both modalities are disabled","Default arguments to empty-then-skip logic in your wrapper instead of invoking forward with Nones"],"exampleFix":"# before\nduration = duration_head(None, None)  # ValueError\n# after\nif video_tokens is None and audio_tokens is None:\n    duration = None\nelse:\n    duration = duration_head(video_tokens, audio_tokens)","handlingStrategy":"type-guard","validationCode":"if video_tokens is None and audio_tokens is None:\n    return None  # skip duration head","typeGuard":"def has_duration_input(video_tokens, audio_tokens) -> bool:\n    return video_tokens is not None or audio_tokens is not None","tryCatchPattern":null,"preventionTips":["Gate the duration-head call on modality availability in the pipeline","Treat both-None as a no-op path, not an error path, in wrappers"],"tags":["ltx","duration-head","api-contract"],"backgroundTag":null,"analyzedSha":"1c6d8d45b3693bfbb32385b410d813a7fd6be216","analyzedAt":"2026-08-14T19:37:18.893Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}