{"record":{"id":"ecf0195dc91b21ba","repo":"microsoft/VibeVoice","slug":"audio-at-index-item-idx-is-too-short-to-be-repre","errorCode":null,"errorMessage":"Audio at index {item_idx} is too short to be represented","messagePattern":"Audio at index (.+?) is too short to be represented","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"vllm_plugin/model.py","lineNumber":886,"sourceCode":"                return 0\n            if isinstance(x, torch.Tensor):\n                # Accept 0-dim or 1-dim scalar-like tensors\n                if x.numel() == 1:\n                    return int(x.item())\n                # If a full tensor is passed accidentally, fall back to its length\n                return int(x.shape[0])\n            return int(x)\n        \n        def get_replacement(item_idx: int):\n            if raw_audio_lengths and item_idx < len(raw_audio_lengths):\n                audio_len = _to_int_len(raw_audio_lengths[item_idx])\n                num_features = max(1, int(np.ceil(audio_len / compress_ratio)))\n            else:\n                # Fallback: estimate for 30 second audio at 24kHz\n                num_features = int(np.ceil(30 * 24000 / compress_ratio))\n            \n            if num_features == 0:\n                raise ValueError(\n                    f\"Audio at index {item_idx} is too short to be represented\"\n                )\n            \n            # Build replacement token sequence:\n            #   <|speech_start|> + N * <|speech_pad|> + <|speech_end|> + \\n\n            # The newline is important for correct prompt structure.\n            newline_id = 198  # '\\n' token\n            if speech_start_id is not None and speech_pad_id is not None and speech_end_id is not None:\n                embed_id = int(speech_pad_id)\n                replacement_ids = [int(speech_start_id)] + [embed_id] * num_features + [int(speech_end_id), newline_id]\n            # Fallback: add audio BOS/EOS boundaries around repeated <|AUDIO|>.\n            elif audio_bos_id is not None and audio_eos_id is not None:\n                embed_id = int(audio_token_id)\n                replacement_ids = [int(audio_bos_id)] + [embed_id] * num_features + [int(audio_eos_id)]\n            else:\n                embed_id = int(audio_token_id)\n                replacement_ids = [embed_id] * num_features\n","sourceCodeStart":868,"sourceCodeEnd":904,"githubUrl":"https://github.com/microsoft/VibeVoice/blob/94da20d98b2fa7688e9cbfaf7692ddb4954f7600/vllm_plugin/model.py#L868-L904","documentation":"Raised inside the prompt-repair helper `get_replacement(item_idx)` used when vLLM needs a stand-in token sequence for an audio item. The helper computes num_features = ceil(audio_len / compress_ratio) (or a 30-second fallback estimate when raw_audio_lengths is missing/short) and refuses to emit an empty replacement if that value comes out 0 — i.e., the audio contributes zero frames and cannot be represented by even a single speech_pad token.","triggerScenarios":"An audio item whose reported raw_audio_lengths entry is 0 (empty/failed decode that still produced a tensor slot); raw_audio_lengths shorter than the number of audio items combined with a degenerate compress_ratio so large that ceil(30*24000/ratio) == 0; NaN/negative length values flowing in from a malformed preprocessing step.","commonSituations":"Batch requests where one clip decoded to zero samples; a preprocessing bug upstream (e.g., ffmpeg produced empty output for a corrupt file but the pipeline continued); mismatch between number of <|AUDIO|> placeholders and supplied audio items causing the fallback branch to run with unexpected values.","solutions":["Filter zero-length audio client-side before batching: drop clips with len(waveform) == 0.","Check that the number of audio items matches the number of audio placeholders in the prompt so raw_audio_lengths aligns with item indices.","Re-encode corrupt source files (ffmpeg -i in.wav out.wav) and retry; verify decode produces nonzero samples.","If you control the plugin, clamp num_features to at least 1 in the fallback branch (mirroring the max(1, ...) already present in the primary branch)."],"exampleFix":"# before\nitems = [a for a in batch]  # may contain zero-sample arrays\n\n# after\nitems = [a for a in batch if a is not None and len(a) > 0]  # drop empty clips","handlingStrategy":"validation","validationCode":"import numpy as np\n\ndef sanitize_batch(clips: list, min_samples: int = 1) -> list:\n    \"\"\"Drop clips too short/empty to occupy at least one frame.\"\"\"\n    ok = [np.asarray(c, dtype=np.float32) for c in clips]\n    ok = [c for c in ok if c.size >= min_samples and np.isfinite(c).all()]\n    if not ok:\n        raise ValueError(\"batch contains no usable audio\")\n    return ok","typeGuard":"def is_representable_audio(wave: np.ndarray, compress_ratio: float) -> bool:\n    import math\n    return len(wave) > 0 and math.ceil(len(wave) / compress_ratio) >= 1","tryCatchPattern":"try:\n    outs = llm.generate(prompts, multi_modal_data=batch)\nexcept ValueError as e:\n    if \"too short to be represented\" in str(e):\n        batch = sanitize_batch(batch); outs = llm.generate(prompts, multi_modal_data=batch)\n    else:\n        raise","preventionTips":["Assert len(waveform) > 0 right after every decode step in your pipeline.","Keep placeholder count and audio item count in lockstep when building prompts.","Log per-item raw lengths before submit so the failing index is identifiable."],"tags":["audio","batching","prompt-construction","data-quality","vllm"],"backgroundTag":null,"analyzedSha":"94da20d98b2fa7688e9cbfaf7692ddb4954f7600","analyzedAt":"2026-08-15T04:12:07.418Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}