{"record":{"id":"c48476eb940abf6c","repo":"cocoindex-io/cocoindex","slug":"cocoindex-run-gpu-in-subprocess-1-with-num-gpus-p","errorCode":null,"errorMessage":"COCOINDEX_RUN_GPU_IN_SUBPROCESS=1 with num_gpus={pool.num_gpus}: subprocess mode does not yet support per-GPU CUDA_VISIBLE_DEVICES. All subprocess calls run on the same GPU regardless of pool assignment. Use in-process mode for multi-GPU support.","messagePattern":"COCOINDEX_RUN_GPU_IN_SUBPROCESS=1 with num_gpus=(.+?): subprocess mode does not yet support per-GPU CUDA_VISIBLE_DEVICES\\. All subprocess calls run on the same GPU regardless of pool assignment\\. Use in-process mode for multi-GPU support\\.","errorType":"console","errorClass":"UserWarning","httpStatus":null,"severity":"warning","filePath":"python/cocoindex/_internal/runner.py","lineNumber":497,"sourceCode":"                ),\n            )\n        finally:\n            await self._release_gpu(gpu_id)\n\n\nGPU = GPURunner(fraction=1.0)\n\n_subprocess_multi_gpu_warned = False\n\n\ndef _warn_subprocess_multi_gpu() -> None:\n    global _subprocess_multi_gpu_warned\n    if _subprocess_multi_gpu_warned:\n        return\n    _subprocess_multi_gpu_warned = True\n    pool = _get_default_gpu_pool()\n    if pool.num_gpus > 1:\n        warnings.warn(\n            f\"COCOINDEX_RUN_GPU_IN_SUBPROCESS=1 with num_gpus={pool.num_gpus}: \"\n            \"subprocess mode does not yet support per-GPU CUDA_VISIBLE_DEVICES. \"\n            \"All subprocess calls run on the same GPU regardless of pool \"\n            \"assignment. Use in-process mode for multi-GPU support.\",\n            UserWarning,\n            stacklevel=4,\n        )\n","sourceCodeStart":479,"sourceCodeEnd":505,"githubUrl":"https://github.com/cocoindex-io/cocoindex/blob/e84aa99b3292c5270a4b313b2a7137ad9ce8ab3b/python/cocoindex/_internal/runner.py#L479-L505","documentation":"When COCOINDEX_RUN_GPU_IN_SUBPROCESS=1, GPU work runs in spawned subprocesses, but that mode cannot set per-GPU CUDA_VISIBLE_DEVICES, so every subprocess lands on the same GPU no matter which GPU the pool assigned. The library emits this UserWarning once (guarded by a module-level flag) whenever subprocess mode is on and the GPU pool has more than one GPU, so results and performance degrade silently otherwise.","triggerScenarios":"Setting the environment variable COCOINDEX_RUN_GPU_IN_SUBPROCESS=1 while the default GPU pool (from _get_default_gpu_pool()) reports num_gpus > 1; emitted from _warn_subprocess_multi_gpu, invoked via run/run_sync_fn on the first GPU-pool-using call.","commonSituations":"Enabling subprocess GPU execution on a multi-GPU machine (e.g. CUDA_VISIBLE_DEVICES lists 2+ GPUs) to isolate CUDA state or work around driver issues, without realizing subprocess mode is single-GPU only.","solutions":["Unset COCOINDEX_RUN_GPU_IN_SUBPROCESS (or set it to 0) so GPU work runs in-process with proper per-GPU CUDA_VISIBLE_DEVICES assignment.","If subprocess isolation is required, restrict the pool to one GPU so the warning is moot and behavior matches expectations.","Ignore the warning only if you intentionally want all subprocess work on a single GPU."],"exampleFix":"// before\nCOCOINDEX_RUN_GPU_IN_SUBPROCESS=1 python app.py   # multi-GPU machine: all subprocesses hit GPU 0\n// after\nunset COCOINDEX_RUN_GPU_IN_SUBPROCESS             # in-process mode honors per-GPU assignment","handlingStrategy":"validation","validationCode":"import os\nif os.environ.get(\"COCOINDEX_RUN_GPU_IN_SUBPROCESS\") == \"1\":\n    n = _get_default_gpu_pool().num_gpus  # or torch.cuda.device_count()\n    if n > 1:\n        del os.environ[\"COCOINDEX_RUN_GPU_IN_SUBPROCESS\"]  # force in-process mode","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Only set COCOINDEX_RUN_GPU_IN_SUBPROCESS on single-GPU machines or pools.","Check pool.num_gpus before opting into subprocess mode.","Keep GPU execution in-process unless you specifically need CUDA state isolation.","Treat this UserWarning as a config bug, not noise — fix the env var at startup."],"tags":["gpu","subprocess","cuda","configuration","python"],"backgroundTag":"invalid-config-value","analyzedSha":"e84aa99b3292c5270a4b313b2a7137ad9ce8ab3b","analyzedAt":"2026-09-08T15:59:19.997Z","contentChangedAt":"2026-09-08T15:59:19.997Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}