sgl-project/sglang · error · ValueError

Number of gpus must be positive

Error message

Number of gpus must be positive

What it means

Raised by the validate() method of the `generate` CLI command when --num-gpus (args.num_gpus) is provided and is <= 0. The value must be a positive integer because it drives distributed launch of the generation workers.

Source

Thrown at python/sglang/multimodal_gen/runtime/entrypoints/cli/generate.py:234

        self.generation_arg_names = self._get_generation_arg_names()

    def _get_init_arg_names(self) -> list[str]:
        """Get names of arguments for DiffGenerator initialization"""
        return ["num_gpus", "tp_size", "sp_size", "model_path"]

    def _get_generation_arg_names(self) -> list[str]:
        """Get names of arguments for generate_video method"""
        return [field.name for field in dataclasses.fields(SamplingParams)]

    def cmd(
        self, args: argparse.Namespace, unknown_args: list[str] | None = None
    ) -> None:
        generate_cmd(args, unknown_args)

    def validate(self, args: argparse.Namespace) -> None:
        """Validate the arguments for this command"""
        if args.num_gpus is not None and args.num_gpus <= 0:
            raise ValueError("Number of gpus must be positive")

        if args.config and not os.path.exists(args.config):
            raise ValueError(f"Config file not found: {args.config}")

    def subparser_init(
        self, subparsers: argparse._SubParsersAction
    ) -> FlexibleArgumentParser:
        generate_parser = subparsers.add_parser(
            "generate",
            help="Run inference on a model",
            usage="sglang generate (--model-path MODEL_PATH_OR_ID --prompt PROMPT) | --config CONFIG_FILE [OPTIONS]",
        )

        generate_parser = add_multimodal_gen_generate_args(generate_parser)

        return cast(FlexibleArgumentParser, generate_parser)

View on GitHub (pinned to 0132848349)

Solutions

1. Check GPU availability: run nvidia-smi; if empty, fix CUDA_VISIBLE_DEVICES or run on a GPU host
2. Pass an explicit positive value, e.g. --num-gpus 1
3. If the count is computed in a script, add a guard that fails early with a clear message when the detected count is 0

Example fix

# before
NUM_GPUS=$(nvidia-smi --list-gpus | wc -l)
sglang generate --num-gpus $NUM_GPUS ...

# after
NUM_GPUS=$(nvidia-smi --list-gpus | wc -l)
[ "$NUM_GPUS" -gt 0 ] || { echo 'no GPUs detected'; exit 1; }
sglang generate --num-gpus $NUM_GPUS ...

if args.num_gpus is not None:
    assert args.num_gpus > 0, "--num-gpus must be positive"

When it happens

Trigger: Running `sglang generate --num-gpus 0` or a negative value (possibly via a config/computed variable that evaluated to 0).

Common situations: Shell scripts computing GPU count from nvidia-smi or CUDA_VISIBLE_DEVICES when no GPUs are visible (count=0), typos, CI runners without GPUs.

Related errors

• --diffusers-kwargs must be valid JSON. Got: {args.diffusers_
• Config file not found: {args.config}
• Config file not found: {args.config}
• error: unrecognized arguments: {' '.join(remaining)}
• Unknown serve backend {name!r}. Available values: {available

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/506b3e4a1dc914f4. Report an issue: GitHub.