sgl-project/sglang · error · ValueError

Invalid precision: {self.precision}. Must be 'int4' or 'nvfp

Error message

Invalid precision: {self.precision}. Must be 'int4' or 'nvfp4'

What it means

NunchakuConfig.__post_init__ rejects a precision that is neither 'int4' nor 'nvfp4'. The first check fires only when group_size was omitted, because the default group size depends on the precision (16 for nvfp4, 64 for int4).

Source

Thrown at python/sglang/multimodal_gen/runtime/layers/quantization/configs/nunchaku_config.py:137

            rank=self.rank,
            act_unsigned=self.act_unsigned,
        )

    def _get_quant_rules(self) -> dict[str, list[str]]:
        if self.model_cls is not None and hasattr(
            self.model_cls, "get_nunchaku_quant_rules"
        ):
            return self.model_cls.get_nunchaku_quant_rules()
        return {}

    def __post_init__(self):
        if self.group_size is None:
            if self.precision == "nvfp4":
                self.group_size = 16
            elif self.precision == "int4":
                self.group_size = 64
            else:
                raise ValueError(
                    f"Invalid precision: {self.precision}. Must be 'int4' or 'nvfp4'"
                )

        if self.precision not in ["int4", "nvfp4"]:
            raise ValueError(
                f"Invalid precision: {self.precision}. Must be 'int4' or 'nvfp4'"
            )

        if self.rank <= 0:
            raise ValueError(f"Rank must be positive, got {self.rank}")

    @classmethod
    def from_dict(cls, config_dict: dict) -> "NunchakuConfig":
        """Create configuration from dictionary."""
        return cls(**config_dict)

    def to_dict(self) -> dict:
        """Convert configuration to dictionary."""

View on GitHub (pinned to 0132848349)

Solutions

Set precision to exactly 'int4' or 'nvfp4'
Fix case/typo in the precision string
If you need a custom group size, provide group_size explicitly — but precision must still be valid

Example fix

# before
cfg = NunchakuConfig(precision="fp8")

# after
cfg = NunchakuConfig(precision="int4", group_size=64)

Defensive patterns

Strategy: validation

Validate before calling

if cfg_kwargs.get("precision") not in ("int4", "nvfp4"):
    raise SystemExit("precision must be 'int4' or 'nvfp4'")
cfg = NunchakuConfig(**cfg_kwargs)

Type guard

def is_valid_nunchaku_precision(p: object) -> bool:
    return isinstance(p, str) and p in ("int4", "nvfp4")

Prevention

Validate precision against the allowed set at CLI/config parse time
Normalize casing early: precision.strip().lower()

When it happens

Trigger: Constructing NunchakuConfig(precision='fp8') or any other string with group_size=None; the inference chain hits the else branch while trying to pick a default group size.

Common situations: Copy-pasting a config from a different quantization library using 'int8'/'fp8'; case mismatch like 'INT4'; passing None precision.

Understand the failure class

Background: "Invalid value" and "allowed values are" config errors: what your library rejected and how to fix it — this error's family across 41 libraries.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/0c9c565d2cc33ff4. Report an issue: GitHub.