sgl-project/sglang · error · ValueError

LTX2DurationHead requires at least one of video_tokens / aud

Error message

LTX2DurationHead requires at least one of video_tokens / audio_tokens.

What it means

LTX2DurationHead.forward predicts clip duration from connector outputs and requires at least one of video_tokens or audio_tokens. With both None there is no input signal to regress a duration from, so the head refuses rather than producing garbage. Pass whichever modality tokens you have (or both).

Source

Thrown at python/sglang/multimodal_gen/runtime/models/adapter/ltx_2_duration_head.py:92

        self.attention_pooler = LTX2DurationAttentionPooler(
            hidden_dim=pooler_hidden_dim,
            num_queries=arch.num_queries,
            num_heads=arch.num_pooler_heads,
        )
        self.mlp_hidden = nn.Linear(
            pooler_hidden_dim * arch.num_queries, arch.mlp_hidden_dim
        )
        self.mlp_out = nn.Linear(arch.mlp_hidden_dim, 1)

    def forward(
        self,
        video_tokens: torch.Tensor | None = None,
        audio_tokens: torch.Tensor | None = None,
    ) -> torch.Tensor:
        """Returns predicted duration in seconds, shape `(batch,)`."""
        if video_tokens is None and audio_tokens is None:
            raise ValueError(
                "LTX2DurationHead requires at least one of video_tokens / audio_tokens."
            )

        # The connector output can arrive in a different dtype than the head.
        head_dtype = self.mlp_out.weight.dtype

        token_groups = []
        if video_tokens is not None:
            token_groups.append(
                self.video_input_proj(video_tokens.to(head_dtype))
                + self.video_modality_emb
            )
        if audio_tokens is not None:
            token_groups.append(
                self.audio_input_proj(audio_tokens.to(head_dtype))
                + self.audio_modality_emb
            )

View on GitHub (pinned to 0132848349)

Solutions

1. Pass video_tokens (and/or audio_tokens) from the LTX2 connector output into the duration head call
2. Check upstream connector forward: verify its returned dict actually contains the token tensors your code reads
3. Add an assert/logging step in the pipeline that fails earlier with which modality is missing
4. If running audio-only or video-only configs, make sure the enabled tower's tokens are routed to the head

Example fix

# before
seconds = duration_head(video_tokens=batch.get("video_tokens"), audio_tokens=batch.get("audio_tokens"))
# after
video_tokens = batch.get("video_tokens")
audio_tokens = batch.get("audio_tokens")
assert video_tokens is not None or audio_tokens is not None, "connector produced no tokens"
seconds = duration_head(video_tokens=video_tokens, audio_tokens=audio_tokens)

assert video_tokens is not None or audio_tokens is not None, "LTX2DurationHead needs at least one token tensor"

def has_duration_inputs(v: torch.Tensor | None, a: torch.Tensor | None) -> bool:
    return v is not None or a is not None

try:
    seconds = head(video_tokens=v, audio_tokens=a)
except ValueError as e:
    if "at least one of video_tokens" in str(e):
        raise RuntimeError(f"connector produced no tokens for batch: {batch_keys}") from e
    raise

When it happens

Trigger: Calling `head.forward()` / `head(video_tokens=None, audio_tokens=None)` — e.g. a pipeline step that forwards connector outputs but both fields were dropped, defaulted to None, or an empty batch dict was passed through.

Common situations: Wiring a new multimodal pipeline where the audio tower is disabled but video tokens were accidentally not propagated; early prototyping with placeholder None arguments; refactors that renamed connector output keys so token lookup returns None.

Related errors

• Replicated Q, K, and V must be provided together.
• predict_num_frames supports a single prediction only, got sh
• recycle_interval must be positive
• attn_sink requires topk_length to be provided as well
• missing value for {a} (expected e.g. `{a} 2,4`)

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/edaeeddc62acb789. Report an issue: GitHub.