sgl-project/sglang · critical · RuntimeError

MiniMax H3 audio encode failed on rank 0: {owner_error}

Error message

MiniMax H3 audio encode failed on rank 0: {owner_error}

What it means

Audio reference encoding runs only on rank 0; its exception (or error string) is broadcast across the replica group. If rank 0 failed and no exception object is available on this rank, a RuntimeError wrapping 'MiniMax H3 audio encode failed on rank 0: <owner_error>' is raised so all ranks fail together.

Source

Thrown at python/sglang/multimodal_gen/runtime/pipelines_core/stages/model_specific_stages/minimax_h3/stages/audio_encoding.py:125

                    module=self.audio_vae,
                ) as audio_vae:
                    assert audio_vae is not None
                    self.audio_vae = audio_vae
                    batch.extra[MINIMAX_H3_REFERENCE_AUDIO_ROWS_EXTRA_KEY] = (
                        self._encode_reference_payload(
                            batch,
                            plan,
                            routed_materials,
                        )
                    )
            except Exception as exc:
                owner_exception = exc
                owner_error = f"{type(exc).__name__}: {exc}"
        owner_error = minimax_h3_replica_broadcast_error(owner_error)
        if owner_error is not None:
            if owner_exception is not None:
                raise owner_exception
            raise RuntimeError(
                f"MiniMax H3 audio encode failed on rank 0: {owner_error}"
            )
        minimax_h3_replica_broadcast_extra(
            batch, MINIMAX_H3_REFERENCE_AUDIO_ROWS_EXTRA_KEY
        )

    def _encode_reference_payload(self, batch: Req, plan, materials) -> dict:
        from sglang.multimodal_gen.runtime.pipelines_core.stages.model_specific_stages.minimax_h3.material_io import (
            minimax_h3_localize_material_uri,
        )
        from sglang.multimodal_gen.runtime.pipelines_core.stages.model_specific_stages.minimax_h3.prequeue import (
            MINIMAX_H3_PROBE_FACTS_EXTRA_KEY,
        )
        from sglang.multimodal_gen.runtime.pipelines_core.stages.model_specific_stages.minimax_h3.reference_encoding import (
            _AudioVAEDeterminismContext,
            minimax_h3_encode_reference_audio_rows,
        )

View on GitHub (pinned to 0132848349)

Solutions

  1. Inspect the rank-0 log for the underlying owner_error text and fix that root cause (corrupt audio file, OOM, missing component)
  2. Verify reference audio files are readable and correctly formatted before submission
  3. Reduce batch/audio reference size if the root cause is memory pressure
Defensive patterns

Strategy: try-catch

Try / catch

try:
    out = stage.forward(batch)
except RuntimeError as e:
    if "audio encode failed on rank 0" in str(e):
        log_rank0_context(); raise

Prevention

When it happens

Trigger: Any exception inside the rank-0 audio tokenizer encode (corrupt reference audio, device OOM, missing weights) surfaces on non-owner ranks as this RuntimeError; on rank 0 the original exception is re-raised instead.

Common situations: Multi-replica (DP) inference where only one replica's audio encoder hits a bad input or OOM; debugging logs on non-zero ranks showing a wrapped error whose root cause is only in the rank-0 log.

Related errors


AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/e6dac8e9fb655487. Report an issue: GitHub.