What causes "unsupported audio item: loaded={loaded_type}, raw={raw_type}" in sgl-project/sglang?

Convert the audio item to a plain str URL, bytes, torch.Tensor, or a dict with a 'url' key before sending it

sgl-project/sglang · error · ValueError

unsupported audio item: loaded={loaded_type}, raw={raw_type}

Error message

unsupported audio item: loaded={loaded_type}, raw={raw_type}

What it means

MiMo-v2 ASR processor rejects an audio item whose loaded value and raw value are both of unrecognized types. The loader failed to produce a usable object and the raw user-supplied item is not a dict, str, bytes, or torch.Tensor, so the processor has no way to fetch or decode the audio.

Source

Thrown at python/sglang/srt/multimodal/processors/mimo_v2_asr.py:236

        contents: List[_Content] = []

        for text_part in text_parts:
            if multimodal_tokens_pattern.match(text_part):
                modality = self.mm_tokens.get_modality_of_token(text_part)
                assert modality is not None

                if modality == Modality.AUDIO:
                    loaded_audio = next(loaded_audio_iter)
                    raw_audio_item = next(raw_audio_iter)

                    if isinstance(loaded_audio, np.ndarray):
                        audio_source = loaded_audio
                    elif isinstance(raw_audio_item, dict):
                        audio_source = raw_audio_item.get("url", loaded_audio)
                    elif isinstance(raw_audio_item, (str, bytes, torch.Tensor)):
                        audio_source = raw_audio_item
                    else:
                        raise ValueError(
                            f"unsupported audio item: loaded={type(loaded_audio).__name__}, "
                            f"raw={type(raw_audio_item).__name__}"
                        )

                    contents.append(
                        _Content(
                            type="audio",
                            content=AudioInput(audio=audio_source),
                        )
                    )
            else:
                if text_part:
                    contents.append(_Content(type="text", content=text_part))

        loop = asyncio.get_running_loop()
        try:
            input_ids, audio_inputs, position_ids, rope_deltas = (
                await loop.run_in_executor(

View on GitHub (pinned to 0132848349)

Solutions

Convert the audio item to a plain str URL, bytes, torch.Tensor, or a dict with a 'url' key before sending it
If using pathlib.Path, wrap it with str(path)
Check that the audio URL is reachable and loads correctly (a failed load leaves loaded_audio in an unusable state)
Upgrade sglang if the loader in your version returns an unsupported type for your audio format

Example fix

// before
{"type":"audio","audio": Path("/data/speech.wav")}
// after
{"type":"audio","audio": "/data/speech.wav"}

Defensive patterns

Strategy: type-guard

Validate before calling

def is_supported_audio(item) -> bool:
    return item is None or isinstance(item, (str, bytes, torch.Tensor)) or (isinstance(item, dict) and "url" in item)

Type guard

import torch
def is_supported_audio(item: object) -> bool:
    if isinstance(item, (str, bytes, torch.Tensor)):
        return True
    return isinstance(item, dict) and isinstance(item.get("url"), str)

Try / catch

try:
    resp = client.process(audio_item)
except ValueError as e:
    if "unsupported audio item" in str(e):
        raise TypeError(f"Bad audio payload: {audio_item!r}") from e
    raise

Prevention

Always send audio as a str URL, bytes, torch.Tensor, or {'url': ...}
Convert Path to str before serialization
Validate payloads against supported types in client wrappers

When it happens

Trigger: Calling process_mm_data_async (i.e. sending an Audio request) with an audio element that is e.g. an int, None, a PIL object, or a custom class; also when the loader returns a non-standard type and the raw item is not one of the four accepted types.

Common situations: Passing a file path as Path instead of str, passing None for a failed URL fetch, serializing audio through a client that converts it to an unsupported type, or version changes in the loading layer returning new object types.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/274fd5170c8b4f8c. Report an issue: GitHub.