Comfy-Org/ComfyUI · error · ValueError

DurationHead requires at least one of video_tokens / audio_t

Error message

DurationHead requires at least one of video_tokens / audio_tokens

What it means

LTX 2.4's DurationHead predicts shot duration from connector tokens; its forward needs at least one of video_tokens (B, T_v, 4096) or audio_tokens (B, T_a, 2048) to pool over. Calling it with both None leaves the attention pooler with an empty sequence, so it raises immediately.

Source

Thrown at comfy/ldm/lightricks/duration_head.py:56

        self.video_input_proj = nn.Linear(video_cross_attention_dim, pooler_hidden_dim)
        self.video_modality_emb = nn.Parameter(torch.empty(pooler_hidden_dim))
        self.audio_input_proj = nn.Linear(audio_cross_attention_dim, pooler_hidden_dim)
        self.audio_modality_emb = nn.Parameter(torch.empty(pooler_hidden_dim))
        self.attention_pooler = AttentionPooler(
            hidden_dim=pooler_hidden_dim, num_queries=num_queries, num_heads=num_pooler_heads)
        self.mlp_hidden = nn.Linear(pooler_hidden_dim * num_queries, mlp_hidden)
        self.mlp_out = nn.Linear(mlp_hidden, 1)

    def forward(self, video_tokens=None, audio_tokens=None):
        """``video_tokens``: (B, T_v, 4096), ``audio_tokens``: (B, T_a, 2048);
        at least one required. Returns duration in seconds, shape (B,)."""
        token_groups = []
        if video_tokens is not None:
            token_groups.append(self.video_input_proj(video_tokens) + self.video_modality_emb)
        if audio_tokens is not None:
            token_groups.append(self.audio_input_proj(audio_tokens) + self.audio_modality_emb)
        if not token_groups:
            raise ValueError("DurationHead requires at least one of video_tokens / audio_tokens")
        pooled = self.attention_pooler(torch.cat(token_groups, dim=1))
        pooled = pooled.reshape(pooled.shape[0], -1)
        hidden = F.gelu(self.mlp_hidden(pooled), approximate="tanh")
        return self.mlp_out(hidden).squeeze(-1).exp()


def normalize_state_dict(sd):
    for prefix in ("model.diffusion_model.duration_head.", "duration_head."):
        stripped = {k[len(prefix):]: v for k, v in sd.items() if k.startswith(prefix)}
        if stripped:
            return stripped
    return sd


def seconds_to_num_frames(seconds, frame_rate, min_seconds, max_seconds, time_scale=8):
    """Convert seconds to a frame count clamped to ``[min_seconds, max_seconds]``
    and snapped (floor) to the VAE's ``8k + 1`` causal temporal grid; snapping
    that undershoots the minimum bumps up to the next grid point instead."""

View on GitHub (pinned to 1c6d8d45b3)

Solutions

1. Pass at least one modality's connector output (usually the caption/video tokens)
2. Skip the duration-head call entirely when both modalities are disabled
3. Default arguments to empty-then-skip logic in your wrapper instead of invoking forward with Nones

Example fix

# before
duration = duration_head(None, None)  # ValueError
# after
if video_tokens is None and audio_tokens is None:
    duration = None
else:
    duration = duration_head(video_tokens, audio_tokens)

if video_tokens is None and audio_tokens is None:
    return None  # skip duration head

def has_duration_input(video_tokens, audio_tokens) -> bool:
    return video_tokens is not None or audio_tokens is not None

When it happens

Trigger: duration_head.forward() / model call with both connector outputs omitted, e.g. running a text-only path through a node that always instantiates the head, or a script that passes tokens=None when a modality is disabled.

Common situations: Disabling both audio and video connectors while keeping the duration head enabled, or refactoring a pipeline where tokens were previously positional and got shifted to None.

Related errors

• The connected model_patch is not an LTX duration head.
• Unsupported spatial_scale {scale}. Choose from {list(mapping
• Either spatial_upsample or temporal_upsample must be True
• Unknown activation function: {act_fn}
• keyframe_idxs holds {keyframe_idxs.shape[2]} tokens, which i

AI-assisted analysis of Comfy-Org/ComfyUI@1c6d8d45b3 (2026-08-14). Data as JSON: /api/errors/acd0be3088673f16. Report an issue: GitHub.