sgl-project/sglang · error · ValueError

--enable-svdquant requires --transformer-weights-path to be

Error message

--enable-svdquant requires --transformer-weights-path to be set

What it means

SVDQuant via Nunchaku does not fetch quantized weights itself; you must point it at the quantized transformer weights directory with --transformer-weights-path. The validator raises when enable_svdquant is set but transformer_weights_path is empty.

Source

Thrown at python/sglang/multimodal_gen/configs/quantization/nunchaku.py:130

        device_count = torch.cuda.device_count()

        unsupported: list[str] = []
        for i in range(device_count):
            major, minor = torch.cuda.get_device_capability(i)
            if major == 9:
                unsupported.append(f"cuda:{i} (SM{major}{minor}, Hopper)")
            elif major not in (8, 12):
                unsupported.append(f"cuda:{i} (SM{major}{minor})")

        if unsupported:
            raise ValueError(
                "Nunchaku SVDQuant is currently only supported on Ampere (SM8x) or SM12x GPUs; "
                f"Unsupported devices: {', '.join(unsupported)}. "
                "Disable it with --enable-svdquant false."
            )

        if not self.transformer_weights_path:
            raise ValueError(
                "--enable-svdquant requires --transformer-weights-path to be set"
            )

        if not is_nunchaku_available():
            raise ValueError(
                "Nunchaku is enabled, but not installed. Please refer to https://nunchaku.tech/docs/nunchaku/installation/installation.html for detailed installation methods."
            )

        if self.quantization_precision not in ("int4", "nvfp4"):
            raise ValueError(
                f"Invalid --quantization-precision: {self.quantization_precision}. "
                "Must be one of: int4, nvfp4"
            )

        if self.quantization_rank <= 0:
            raise ValueError(
                f"Invalid --quantization-rank: {self.quantization_rank}. Must be > 0"
            )

View on GitHub (pinned to 0132848349)

Solutions

Download/prepare the SVDQuant-quantized transformer weights (e.g. from the nunchaku/hub releases for your model) and pass --transformer-weights-path /path/to/svdquant-weights
Verify the path exists and contains the expected quantized transformer files
If you did not intend to use SVDQuant, drop --enable-svdquant (or set it to false)

Example fix

# before
sglang.launch_server ... --enable-svdquant true

# after
sglang.launch_server ... --enable-svdquant true \
  --transformer-weights-path /models/qwen-image-svdquant-int4

Defensive patterns

Strategy: validation

Validate before calling

import os
if cfg.enable_svdquant:
    assert cfg.transformer_weights_path and os.path.isdir(cfg.transformer_weights_path), \
        "--enable-svdquant requires an existing --transformer-weights-path directory"

Type guard

def svdquant_config_complete(cfg) -> bool:
    import os
    return (not cfg.enable_svdquant) or bool(cfg.transformer_weights_path and os.path.exists(cfg.transformer_weights_path))

Prevention

Always pair --enable-svdquant with --transformer-weights-path
Download the official SVDQuant weight release for your model ahead of time
Add a preflight check that the weights dir is non-empty before launch

When it happens

Trigger: Starting with --enable-svdquant true without also passing --transformer-weights-path; raised from _validate during resolve_runtime_config at startup, before any model loads.

Common situations: Assuming the flag alone switches quantization and the hub checkpoint is enough; typos in the weights-path argument leaving it None; config templates that enable svdquant but omit the weights path field.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/84a4ec892737aa30. Report an issue: GitHub.