huggingface/transformers · error · TypeError

Incorrect format used for `audio`. Should be a numpy array o

Error message

Incorrect format used for `audio`. Should be a numpy array or a `str`: an `http(s)://` URL, a local file path, or a base64-encoded string (optionally wrapped as a `data:...` URI).

What it means

Identical dtype gate to the BF16 experts path, applied in `deepgemm_fp8_fp4_experts_forward`: hidden states entering the FP8/FP4 grouped experts must be bfloat16, because the per-token cast and the grouped kernels' dequant output are bf16. fp16/fp32 activations are rejected before kernel load.

Source

Thrown at src/transformers/audio_utils.py:224

            `data:<media-type>;base64,` URI).
        sampling_rate (`int`, *optional*, defaults to 16000):
            The sampling rate to be used when loading the audio. It should be same as the
            sampling rate the model you will be using further was trained with.
        timeout (`float`, *optional*):
            The timeout value in seconds for the URL request.
        backend (`str`, *optional*, defaults to `"auto"`):
            Decoding backend: `"auto"` uses torchcodec when available (>=0.3.0) and falls back to
            librosa; `"torchcodec"`, `"librosa"` or `"torchaudio"` force that backend (and error if it
            is missing). `"torchaudio"` decodes with `torchaudio.load` and resamples with
            `torchaudio.functional.resample` (matches serving stacks such as sglang bit-for-bit).

    Returns:
        `np.ndarray`: A numpy array representing the audio.
    """
    if isinstance(audio, np.ndarray):
        return audio
    if not isinstance(audio, str):
        raise TypeError(
            "Incorrect format used for `audio`. Should be a numpy array or a `str`: an `http(s)://` URL, "
            "a local file path, or a base64-encoded string (optionally wrapped as a `data:...` URI)."
        )

    # torchcodec handles audio/video; librosa only plain audio. `backend` lets callers pin one.
    if backend == "auto":
        resolved_backend = (
            "torchcodec" if is_torchcodec_available() and version.parse("0.3.0") <= TORCHCODEC_VERSION else "librosa"
        )
    elif backend in ("torchcodec", "librosa", "torchaudio"):
        resolved_backend = backend
    else:
        raise ValueError(f"Unknown backend {backend!r}; expected 'auto', 'torchcodec', 'librosa', or 'torchaudio'.")
    # soundfile-based backends (librosa / torchaudio) cannot decode the video-ish formats below.
    use_torchcodec = resolved_backend == "torchcodec"

    # 1. Identify the format from the source string (extension / `data:` media type), without fetching.
    filetype = _format_from_source(audio)

View on GitHub (pinned to a597f97485)

Solutions

1. Load/run the model in bfloat16 (`torch_dtype=torch.bfloat16`)
2. Cast at the boundary: `hidden_states = hidden_states.to(torch.bfloat16)` before the experts
3. Verify autocast dtype is bf16 when the MoE forward runs under autocast

Example fix

# before
out = model(input_ids)  # model in fp16 -> ValueError

# after
model = AutoModelForCausalLM.from_pretrained(ckpt, torch_dtype=torch.bfloat16)
out = model(input_ids)

if hidden_states.dtype != torch.bfloat16:
    hidden_states = hidden_states.to(torch.bfloat16)

def is_bf16(t: torch.Tensor) -> bool:
    return t.dtype == torch.bfloat16

When it happens

Trigger: FP8/FP4 DeepGEMM experts forward with `hidden_states.dtype != torch.bfloat16` — typically a model loaded in float16 or float32, or a preceding layer that upcasts to float32 (e.g. RMSNorm in fp32 without a downcast).

Common situations: `torch_dtype=torch.float16` checkpoints on new stacks; custom norm code that leaves `.float()` activations; mixed autocast configurations where the MoE block runs outside the autocast region.

Related errors

• function {activation_string} not found in ACT2FN mapping {li
• min should be < max (got min: {min}, max: {max})
• not supported filetype
• Incorrect audio source. Must be a valid URL starting with `h
• Unknown backend {backend!r}; expected 'auto', 'torchcodec',

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/45a68e5be42e96fd. Report an issue: GitHub.