sgl-project/sglang · error · ValueError

Rank must be positive, got {self.rank}

Error message

Rank must be positive, got {self.rank}

What it means

NunchakuConfig.__post_init__ requires rank to be a positive integer; rank <= 0 (including 0 or negatives, and None comparisons depending on type) raises immediately.

Source

Thrown at python/sglang/multimodal_gen/runtime/layers/quantization/configs/nunchaku_config.py:147

    def __post_init__(self):
        if self.group_size is None:
            if self.precision == "nvfp4":
                self.group_size = 16
            elif self.precision == "int4":
                self.group_size = 64
            else:
                raise ValueError(
                    f"Invalid precision: {self.precision}. Must be 'int4' or 'nvfp4'"
                )

        if self.precision not in ["int4", "nvfp4"]:
            raise ValueError(
                f"Invalid precision: {self.precision}. Must be 'int4' or 'nvfp4'"
            )

        if self.rank <= 0:
            raise ValueError(f"Rank must be positive, got {self.rank}")

    @classmethod
    def from_dict(cls, config_dict: dict) -> "NunchakuConfig":
        """Create configuration from dictionary."""
        return cls(**config_dict)

    def to_dict(self) -> dict:
        """Convert configuration to dictionary."""
        return {
            "precision": self.precision,
            "rank": self.rank,
            "group_size": self.group_size,
            "act_unsigned": self.act_unsigned,
            "transformer_weights_path": self.transformer_weights_path,
        }

    @classmethod
    def from_pretrained(cls, model_path: str) -> Optional["NunchakuConfig"]:

View on GitHub (pinned to 0132848349)

Solutions

1. Pass the actual distributed rank (>= 1 for this config, or 1 for single-GPU)
2. Default rank to 1 when running without torch.distributed initialization

Example fix

# before
cfg = NunchakuConfig(precision="int4", rank=dist.get_rank() - 1)

# after
import os
cfg = NunchakuConfig(precision="int4", rank=int(os.environ.get("RANK", "1")) or 1)

rank = int(os.environ.get("RANK", "1"))
if rank <= 0:
    rank = 1
cfg = NunchakuConfig(precision="int4", rank=rank)

def is_valid_rank(rank: object) -> bool:
    return isinstance(rank, int) and not isinstance(rank, bool) and rank > 0

When it happens

Trigger: NunchakuConfig(precision='int4', rank=0) or rank=-1, typically when rank is taken from an unset env var / CLI default of 0.

Common situations: rank computed as world_size - 1 for single-process runs (0); forgetting to initialize a distributed rank; passing device index instead of rank.

Understand the failure class

Background: "Invalid value" and "allowed values are" config errors: what your library rejected and how to fix it — this error's family across 41 libraries.

Related errors

• Invalid precision: {self.precision}. Must be 'int4' or 'nvfp
• Ring Attention requires one of the ring-capable backends ({'
• bad compress_ratio {compress_ratio}
• The requested FlashAttention forward configuration exceeds S
• flashinfer_sparse_mla supports only GLM DSA with FP8 KV cach

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/b692645f174ab30d. Report an issue: GitHub.