{"record":{"id":"9b9cd2ada97c4209","repo":"sgl-project/sglang","slug":"invalid-number-of-gpus-requested-n-available","errorCode":null,"errorMessage":"Invalid number of GPUs requested: {N} (available: {num_devices})","messagePattern":"Invalid number of GPUs requested: (.+?) \\(available: (.+?)\\)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/communication/mp.py","lineNumber":163,"sourceCode":"        if rank != 0:\n            sys.stdout = open(os.devnull, \"w\")\n        torch.cuda.set_device(rank)\n        return sys.exit(inner())\n    assert pid_key not in os.environ\n    if name != \"__main__\":\n        return logger.warning(\n            f\"{file} can not directly run with `pytest`. \"\n            \"Use `python` to invoke it, which will internally relaunch it \"\n            \"under torchrun for each requested number of GPUs.\"\n        )\n    num_devices = torch.cuda.device_count()\n    override, forwarded_args = _extract_num_gpus_override(sys.argv[1:])\n    if override is not None:\n        logger.info(f\"--num-gpu override: running only with {override}\")\n        num_gpus = override\n        for N in num_gpus:\n            if N <= 1 or N > num_devices:\n                raise ValueError(\n                    f\"Invalid number of GPUs requested: {N} \"\n                    f\"(available: {num_devices})\"\n                )\n    os.environ[env_key] = \"1\"\n    os.environ[pid_key] = str(os.getpid())\n    os.environ.setdefault(\"OMP_NUM_THREADS\", \"1\")\n    os.environ.setdefault(\"GLOO_SOCKET_IFNAME\", \"lo\")  # single-machine setup\n    # Unbuffered child stdout: when a worker is killed on timeout, pytest's\n    # block-buffered progress output is otherwise lost or flushed out of\n    # order into the CI log, making it impossible to tell which test hung.\n    os.environ.setdefault(\"PYTHONUNBUFFERED\", \"1\")\n    signal.signal(signal.SIGINT, signal.default_int_handler)\n    runnable: List[int] = []\n    for N in sorted(num_gpus):\n        assert N > 1\n        if N > num_devices:\n            logger.warning(f\"Skipping {kind} with {N} GPUs ({num_devices} available)\")\n            continue","sourceCodeStart":145,"sourceCodeEnd":181,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/communication/mp.py#L145-L181","documentation":"After resolving a --num-gpu/--num-gpus override, multigpu_launch validates every requested GPU count N: it must be >= 2 (single-GPU makes multi-process launch pointless) and <= the number of visible CUDA devices. Otherwise it aborts before spawning workers.","triggerScenarios":"Running `--num-gpu 8` on a 4-GPU node, or `--num-gpu 1`/`--num-gpu 0`; also triggered when CUDA_VISIBLE_DEVICES hides devices so torch.cuda.device_count() is smaller than the requested N.","commonSituations":"CI boxes with fewer GPUs than the developer's machine; docker runs that map only a subset of GPUs; accidentally passing a GPU index (e.g. 3) instead of a count.","solutions":["Request a count within [2, torch.cuda.device_count()]: check with `python -c \"import torch; print(torch.cuda.device_count())\"`","Fix CUDA_VISIBLE_DEVICES so the intended number of GPUs is visible before launching","Drop the --num-gpu flag to run the default all-GPU sweep"],"exampleFix":"# before\nCUDA_VISIBLE_DEVICES=0 pytest test_mp.py --num-gpu=4\n# after\nCUDA_VISIBLE_DEVICES=0,1,2,3 pytest test_mp.py --num-gpu=4","handlingStrategy":"validation","validationCode":"import torch\nn = torch.cuda.device_count()\nassert all(1 < N <= n for N in requested), f'request {requested} vs {n} visible GPUs'","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Print torch.cuda.device_count() before multi-GPU test runs","Remember the value is a COUNT (>=2), not a GPU index"],"tags":["multi-gpu","cli-validation","cuda-devices"],"backgroundTag":"invalid-gpu-count-request","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}