sgl-project/sglang · error · RuntimeError

SGLANG_RUST_SERVER=1: no native Rust MM pipeline for model_t

Error message

SGLANG_RUST_SERVER=1: no native Rust MM pipeline for model_type={scheduler.model_config.hf_config.model_type!r} (supported: {', '.join(supported)}; images only). Unset SGLANG_RUST_SERVER to serve this model.

What it means

With SGLANG_RUST_SERVER=1 and a multimodal model, launch requires a native Rust MM preprocessing pipeline for the model's model_type. Only a limited set of image-only families (NATIVE_MM_FAMILIES) are supported; anything else raises RuntimeError at startup.

Source

Thrown at python/sglang/srt/managers/rust_server.py:450

            # preempt the scheduler loop and inflate inter-token latency.
            if server_cores is not None:
                try:
                    os.sched_setaffinity(0, set(server_cores))
                except OSError as e:
                    logger.warning(
                        "rust server: cannot confine mm threads to server cores: %s", e
                    )
            mm_host = NativeMmHost(
                server_args=server_args,
                model_config=scheduler.model_config,
                processor=scheduler.processor,
            )
            mm_spec = mm_host.resolve_native_spec()
            if mm_spec is None:
                supported = sorted(
                    set(chain.from_iterable(f.model_types for f in NATIVE_MM_FAMILIES))
                )
                raise RuntimeError(
                    "SGLANG_RUST_SERVER=1: no native Rust MM pipeline for "
                    f"model_type={scheduler.model_config.hf_config.model_type!r} "
                    f"(supported: {', '.join(supported)}; "
                    "images only). Unset SGLANG_RUST_SERVER to serve this model."
                )
            server.start_mm_workers(cls._build_mm_spec(mm_spec), mm_host.mm_workers)

        # Narrow the scheduler thread only after the server threads are launched.
        if launch_cores is not None:
            try:
                # pid 0 == this thread (the scheduler event-loop / launch thread).
                os.sched_setaffinity(0, set(launch_cores))
            except OSError as e:
                logger.warning("rust server: cannot pin scheduler launch thread: %s", e)

        # Under DP every rank runs its own server on its own port, so the rank is
        # what tells two otherwise identical startup lines apart.
        dp_note = (

View on GitHub (pinned to 0132848349)

Solutions

1. Unset SGLANG_RUST_SERVER and use the standard Python server
2. Use a supported image-only model family (see the error's supported list)
3. Upgrade SGLang — native MM family coverage grows over time

Example fix

# before
SGLANG_RUST_SERVER=1 python -m sglang.launch_server --model Qwen2-VL...
# after
python -m sglang.launch_server --model Qwen2-VL...

model_type = model_config.hf_config.model_type
supported = {mt for f in NATIVE_MM_FAMILIES for mt in f.model_types}
if os.environ.get('SGLANG_RUST_SERVER') and model_type not in supported:
    os.environ.pop('SGLANG_RUST_SERVER', None)

try:
    launch(...)
except RuntimeError as e:
    if 'no native Rust MM pipeline' in str(e):
        os.environ.pop('SGLANG_RUST_SERVER', None); launch(...)
    raise

When it happens

Trigger: Serving a VLM whose model_type is not in any NATIVE_MM_FAMILIES entry (e.g. audio/video models or unsupported architectures) with the rust ingress enabled.

Common situations: Trying the rust fast-path on a newly released or video/audio-capable VLM; enabling SGLANG_RUST_SERVER globally via env then launching an MM model.

Related errors

• No processor registered for architecture: {hf_config.archite
• SGLANG_RUST_SERVER does not yet apply --preferred-sampling-p
• Model vocab_size ({vocab_size}) exceeds MM_PAD_SHIFT_VALUE (
• {selection_error}{component_suffix}
• No compatible attention backend is available{component_suffi

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/ed83aeb7bc8b13d7. Report an issue: GitHub.