huggingface/transformers · error · RuntimeError

The audio source is a '{filetype}' file, which librosa canno

Error message

The audio source is a '{filetype}' file, which librosa cannot decode. {_NEEDS_TORCHCODEC}

What it means

The Mega MoE forward is a Blackwell FP4-only fused path: it requires expert weights packed as FP4 stored in int8 (`gate_up_proj.dtype == torch.int8`). A non-int8 dtype means the checkpoint is not NVFP4-quantized (misconfigured or an FP8 checkpoint), so it raises `NotImplementedError` pointing you at the regular 'deepgemm' dispatch for FP8 experts. The `_assert_sm100_requirements` call just before doubles as the SM100 gate since Mega MoE weights are always FP4.

Source

Thrown at src/transformers/audio_utils.py:245

        )

    # torchcodec handles audio/video; librosa only plain audio. `backend` lets callers pin one.
    if backend == "auto":
        resolved_backend = (
            "torchcodec" if is_torchcodec_available() and version.parse("0.3.0") <= TORCHCODEC_VERSION else "librosa"
        )
    elif backend in ("torchcodec", "librosa", "torchaudio"):
        resolved_backend = backend
    else:
        raise ValueError(f"Unknown backend {backend!r}; expected 'auto', 'torchcodec', 'librosa', or 'torchaudio'.")
    # soundfile-based backends (librosa / torchaudio) cannot decode the video-ish formats below.
    use_torchcodec = resolved_backend == "torchcodec"

    # 1. Identify the format from the source string (extension / `data:` media type), without fetching.
    filetype = _format_from_source(audio)
    # 2. With librosa as the only backend, fail fast and clearly on a format it cannot decode.
    if not use_torchcodec and filetype in TORCHCODEC_ONLY_FILETYPES:
        raise RuntimeError(
            f"The audio source is a '{filetype}' file, which librosa cannot decode. {_NEEDS_TORCHCODEC}"
        )

    # 3. Resolve to local path or bytes; sniff format for raw base64 payloads before passing to librosa.
    source = _resolve_audio_source(audio, timeout=timeout)
    if not use_torchcodec and filetype is None and isinstance(source, bytes):
        try:
            filetype = get_audio_filetype(source)
        except ValueError:
            filetype = None
        if filetype in TORCHCODEC_ONLY_FILETYPES:
            raise RuntimeError(
                f"The audio source is a '{filetype}' file, which librosa cannot decode. {_NEEDS_TORCHCODEC}"
            )

    # 4. Decode with the selected backend (`requires_backends` raises a clear error if it is missing).
    if use_torchcodec:
        requires_backends(load_audio, ["torchcodec"])

View on GitHub (pinned to a597f97485)

Solutions

Use an NVFP4-quantized checkpoint (int8-packed weights) with `deepgemm_megamoe` on SM100+
For FP8 experts, use `set_experts_implementation('deepgemm')` as the message suggests
Fix config scripts so experts_implementation is chosen per checkpoint quantization format

Example fix

# before
# FP8 checkpoint
model.set_experts_implementation("deepgemm_megamoe")  # -> NotImplementedError

# after
model.set_experts_implementation("deepgemm")  # FP8 experts on SM100
# or run an NVFP4 checkpoint with "deepgemm_megamoe"

Defensive patterns

Strategy: validation

Validate before calling

gate_w = next(m.parameters() for n, m in model.named_modules() if "gate_up_proj" in n)
if gate_w.dtype != torch.int8:  # not NVFP4-packed
    impl = "deepgemm" if gate_w.dtype == torch.float8_e4m3fn else "grouped_mm"
    # set impl accordingly; megamoe is FP4-only

Type guard

def is_fp4_packed(t: torch.Tensor) -> bool:
    return t.dtype == torch.int8

Try / catch

try:
    out = model(input_ids)
except NotImplementedError as e:
    if "FP4-packed expert weights" in str(e):
        model.set_experts_implementation("deepgemm")
        out = model(input_ids)
    else:
        raise

Prevention

Choose experts_implementation from the checkpoint's quantization format, not a global default
Reserve deepgemm_megamoe for NVFP4 checkpoints on SM100+

When it happens

Trigger: Running `experts_implementation='deepgemm_megamoe'` on an FP8-quantized model (weights float8_e4m3fn) or an unquantized bf16 model, instead of an NVFP4 checkpoint.

Common situations: Copy-pasting the megamoe dispatch flag from a B200 FP4 recipe onto an FP8 DeepSeek model; config-driven scripts that set experts_implementation globally across a fleet of differently-quantized models.

Related errors

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/67ebfb3ed938ad4a. Report an issue: GitHub.