microsoft/VibeVoice · error · ValueError

Audio at index {item_idx} is too short to be represented

Error message

Audio at index {item_idx} is too short to be represented

What it means

Raised inside the prompt-repair helper `get_replacement(item_idx)` used when vLLM needs a stand-in token sequence for an audio item. The helper computes num_features = ceil(audio_len / compress_ratio) (or a 30-second fallback estimate when raw_audio_lengths is missing/short) and refuses to emit an empty replacement if that value comes out 0 — i.e., the audio contributes zero frames and cannot be represented by even a single speech_pad token.

Source

Thrown at vllm_plugin/model.py:886

                return 0
            if isinstance(x, torch.Tensor):
                # Accept 0-dim or 1-dim scalar-like tensors
                if x.numel() == 1:
                    return int(x.item())
                # If a full tensor is passed accidentally, fall back to its length
                return int(x.shape[0])
            return int(x)
        
        def get_replacement(item_idx: int):
            if raw_audio_lengths and item_idx < len(raw_audio_lengths):
                audio_len = _to_int_len(raw_audio_lengths[item_idx])
                num_features = max(1, int(np.ceil(audio_len / compress_ratio)))
            else:
                # Fallback: estimate for 30 second audio at 24kHz
                num_features = int(np.ceil(30 * 24000 / compress_ratio))
            
            if num_features == 0:
                raise ValueError(
                    f"Audio at index {item_idx} is too short to be represented"
                )
            
            # Build replacement token sequence:
            #   <|speech_start|> + N * <|speech_pad|> + <|speech_end|> + \n
            # The newline is important for correct prompt structure.
            newline_id = 198  # '\n' token
            if speech_start_id is not None and speech_pad_id is not None and speech_end_id is not None:
                embed_id = int(speech_pad_id)
                replacement_ids = [int(speech_start_id)] + [embed_id] * num_features + [int(speech_end_id), newline_id]
            # Fallback: add audio BOS/EOS boundaries around repeated <|AUDIO|>.
            elif audio_bos_id is not None and audio_eos_id is not None:
                embed_id = int(audio_token_id)
                replacement_ids = [int(audio_bos_id)] + [embed_id] * num_features + [int(audio_eos_id)]
            else:
                embed_id = int(audio_token_id)
                replacement_ids = [embed_id] * num_features

View on GitHub (pinned to 94da20d98b)

Solutions

1. Filter zero-length audio client-side before batching: drop clips with len(waveform) == 0.
2. Check that the number of audio items matches the number of audio placeholders in the prompt so raw_audio_lengths aligns with item indices.
3. Re-encode corrupt source files (ffmpeg -i in.wav out.wav) and retry; verify decode produces nonzero samples.
4. If you control the plugin, clamp num_features to at least 1 in the fallback branch (mirroring the max(1, ...) already present in the primary branch).

Example fix

# before
items = [a for a in batch]  # may contain zero-sample arrays

# after
items = [a for a in batch if a is not None and len(a) > 0]  # drop empty clips

import numpy as np

def sanitize_batch(clips: list, min_samples: int = 1) -> list:
    """Drop clips too short/empty to occupy at least one frame."""
    ok = [np.asarray(c, dtype=np.float32) for c in clips]
    ok = [c for c in ok if c.size >= min_samples and np.isfinite(c).all()]
    if not ok:
        raise ValueError("batch contains no usable audio")
    return ok

def is_representable_audio(wave: np.ndarray, compress_ratio: float) -> bool:
    import math
    return len(wave) > 0 and math.ceil(len(wave) / compress_ratio) >= 1

try:
    outs = llm.generate(prompts, multi_modal_data=batch)
except ValueError as e:
    if "too short to be represented" in str(e):
        batch = sanitize_batch(batch); outs = llm.generate(prompts, multi_modal_data=batch)
    else:
        raise

When it happens

Trigger: An audio item whose reported raw_audio_lengths entry is 0 (empty/failed decode that still produced a tensor slot); raw_audio_lengths shorter than the number of audio items combined with a degenerate compress_ratio so large that ceil(30*24000/ratio) == 0; NaN/negative length values flowing in from a malformed preprocessing step.

Common situations: Batch requests where one clip decoded to zero samples; a preprocessing bug upstream (e.g., ffmpeg produced empty output for a corrupt file but the pipeline continued); mismatch between number of <|AUDIO|> placeholders and supplied audio items causing the fallback branch to run with unexpected values.

Related errors

• Unsupported audio data type: {type(data)}
• Audio duration ({duration_sec:.1f}s) exceeds the configured
• Sample index {sample_idx} exceeds batch size {self.batch_siz
• load_audio_bytes_use_ffmpeg requires resample=True
• Audio input is required for ASR processing

AI-assisted analysis of microsoft/VibeVoice@94da20d98b (2026-08-15). Data as JSON: /api/errors/ecf0195dc91b21ba. Report an issue: GitHub.