{"record":{"id":"7e6ca13c06c0ef14","repo":"microsoft/VibeVoice","slug":"unsupported-audio-data-type-type-data","errorCode":null,"errorMessage":"Unsupported audio data type: {type(data)}","messagePattern":"Unsupported audio data type: (.+?)","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"vllm_plugin/inputs.py","lineNumber":82,"sourceCode":"        data = data[0]\n\n    audio_waveform = None\n    \n    if isinstance(data, str):\n        # Load from file path\n        audio_waveform = load_audio(data)\n        \n    elif isinstance(data, bytes):\n        # Decode bytes directly via ffmpeg stdin pipe to avoid temp-file IO\n        audio_waveform, _sr = load_audio_bytes_use_ffmpeg(data, resample=True, target_sr=24000)\n        normalizer = AudioNormalizer()\n        audio_waveform = normalizer(audio_waveform)\n                \n    elif isinstance(data, np.ndarray):\n        # Already loaded numpy array\n        audio_waveform = data\n    else:\n        raise ValueError(f\"Unsupported audio data type: {type(data)}\")\n\n    # Validate audio duration before tensor conversion to catch OOM early\n    duration_sec = len(audio_waveform) / 24000\n    if duration_sec > _MAX_AUDIO_DURATION:\n        raise ValueError(\n            f\"Audio duration ({duration_sec:.1f}s) exceeds the configured \"\n            f\"limit ({_MAX_AUDIO_DURATION:.0f}s). Set the \"\n            f\"VIBEVOICE_MAX_AUDIO_DURATION environment variable to adjust \"\n            f\"this limit, or use shorter audio.\"\n        )\n\n    # Convert to tensor\n    audio_tensor = torch.from_numpy(audio_waveform).float()\n    audio_length = audio_tensor.shape[0]\n    \n    return MultiModalInputs({\n        \"audio\": audio_tensor,\n        \"audio_length\": audio_length","sourceCodeStart":64,"sourceCodeEnd":100,"githubUrl":"https://github.com/microsoft/VibeVoice/blob/94da20d98b2fa7688e9cbfaf7692ddb4954f7600/vllm_plugin/inputs.py#L64-L100","documentation":"Raised by the vLLM multimodal input parser for VibeVoice when the audio payload is not one of the supported types. The branch chain in vllm_plugin/inputs.py:82 only handles a path/str (via load_audio), bytes (via ffmpeg stdin decode + AudioNormalizer), and numpy.ndarray (used as-is). Any other object type falls through to this ValueError before any tensor conversion happens.","triggerScenarios":"Passing audio as a torch.Tensor, a Python list of floats, a dict (e.g. {'audio': ...}), a URL, a BytesIO/file object, or None in the 'audio' field of a multimodal input dict. Also triggered by passing a numpy array subclass that fails isinstance(data, np.ndarray) is unlikely, but wrapping audio in an extra layer (list of arrays for a batch) hits it.","commonSituations":"Callers coming from other vLLM multimodal models (e.g. Qwen-Audio) that accept torch tensors or pre-encoded features; sending JSON payloads where audio arrives as base64 str without decoding; accidentally sending a tuple of (waveform, sample_rate) returned by librosa/torchaudio load instead of the waveform alone.","solutions":["Convert the payload before submitting: waveforms to numpy float32 mono at 24 kHz, or raw bytes of any audio file, or a file path string.","If you have a torch.Tensor, call `tensor.numpy()` (after `.cpu().detach()`); if you have (waveform, sr) from a loader, pass only the waveform and resample to 24000 first.","If base64 audio arrives from an API, decode with base64.b64decode(...) and send the resulting bytes.","If a batch, submit each array as a separate audio item rather than a list under one item."],"exampleFix":"// before\ninputs = {\"audio\": (waveform, 24000)}  # tuple from torchaudio.load\n\n# after\nwaveform = torchaudio.functional.resample(waveform, orig_sr, 24000)\ninputs = {\"audio\": waveform.numpy()}  # bare np.ndarray, 24 kHz mono","handlingStrategy":"type-guard","validationCode":"import numpy as np\ndef coerce_audio(data):\n    if isinstance(data, str):\n        return data                      # path: handled by load_audio\n    if isinstance(data, (bytes, bytearray)):\n        return bytes(data)               # ffmpeg stdin path\n    if isinstance(data, np.ndarray) and data.ndim == 1:\n        return data\n    if str(type(data)) == \"<class 'torch.Tensor'>\":\n        return data.detach().cpu().numpy()  # common caller mistake\n    raise TypeError(f\"Convert audio to str path / bytes / 1-D np.ndarray, got {type(data)}\")","typeGuard":"def is_supported_audio(data) -> bool:\n    return isinstance(data, (str, bytes, bytearray)) or (\n        isinstance(data, np.ndarray) and data.ndim == 1\n    )","tryCatchPattern":"try:\n    mm = plugin_input_parser(data)\nexcept ValueError as e:\n    if \"Unsupported audio data type\" in str(e):\n        data = coerce_audio(data)  # then retry once\n    else:\n        raise","preventionTips":["Standardize on one canonical client-side audio format (float32 mono 24 kHz np.ndarray) across your codebase.","Log type(data) before every multimodal submit during integration testing.","Decode base64/URL audio at the API boundary, never inside model calls."],"tags":["audio","input-validation","multimodal","vllm","type-error"],"backgroundTag":null,"analyzedSha":"94da20d98b2fa7688e9cbfaf7692ddb4954f7600","analyzedAt":"2026-08-15T04:12:07.418Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}