WZMIAOMIAO/deep-learning-for-image-processing · critical · EnvironmentError

not find GPU device for training.

Error message

not find GPU device for training.

What it means

train_multi_GPU.py requires an actual CUDA device: it builds device = torch.device(opt.device) and raises EnvironmentError('not find GPU device for training.') if 'cuda' is not in device.type. Multi-GPU DDP training cannot proceed on CPU.

Source

Thrown at pytorch_object_detection/yolov3_spp/train_multi_GPU.py:29

from models import *
from build_utils.datasets import *
from build_utils.utils import *
from train_utils import train_eval_utils as train_util
from train_utils import get_coco_api_from_dataset, init_distributed_mode, torch_distributed_zero_first


def main(opt, hyp):
    # 初始化各进程
    init_distributed_mode(opt)

    if opt.rank in [-1, 0]:
        print(opt)
        print('Start Tensorboard with "tensorboard --logdir=runs", view at http://localhost:6006/')
        tb_writer = SummaryWriter(comment=opt.name)

    device = torch.device(opt.device)
    if "cuda" not in device.type:
        raise EnvironmentError("not find GPU device for training.")

    # 使用DDP后会对每个device上的gradients取均值，所以需要放大学习率
    hyp["lr0"] *= max(1., opt.world_size * opt.batch_size / 64)

    wdir = "weights" + os.sep  # weights dir
    best = wdir + "best.pt"
    results_file = "results{}.txt".format(datetime.datetime.now().strftime("%Y%m%d-%H%M%S"))

    cfg = opt.cfg
    data = opt.data
    epochs = opt.epochs
    batch_size = opt.batch_size
    # accumulate n times before optimizer update (bs 64)
    accumulate = max(round(64 / (opt.world_size * opt.batch_size)), 1)
    weights = opt.weights  # initial training weights
    imgsz_train = opt.img_size
    imgsz_test = opt.img_size  # test image sizes
    multi_scale = opt.multi_scale

View on GitHub (pinned to 1ec3fe6f37)

Solutions

1. Verify GPUs are visible: python -c "import torch; print(torch.cuda.is_available(), torch.cuda.device_count())"
2. Install the CUDA-enabled torch build matching your CUDA driver (see pytorch.org install matrix)
3. Pass --device cuda:0 (and correct --device per rank) or run on a GPU machine

Example fix

// before
python train_multi_GPU.py --device cpu ...
// after
python train_multi_GPU.py --device cuda:0 ...  # after verifying torch.cuda.is_available() == True

import torch
assert torch.cuda.is_available() and torch.cuda.device_count() > 0, \
    'No CUDA device: install CUDA-enabled torch or run on a GPU machine'

try:
    device = torch.device(opt.device)
    if 'cuda' not in device.type:
        raise EnvironmentError('not find GPU device for training.')
except EnvironmentError:
    raise SystemExit('Pass --device cuda:N on a machine with torch.cuda.is_available()==True')

When it happens

Trigger: Running train_multi_GPU.py with --device cpu, or --device cuda:0 while torch.cuda.is_available() is False (no GPU, drivers/CUDA not installed, or a CPU-only torch build).

Common situations: Forgetting the --device flag defaults to cpu; running on a machine/VM without GPUs; installing the wrong torch wheel (cpu-only); CUDA_VISIBLE_DEVICES hiding all GPUs.

Related errors

• not find GPU device for training.
• not find GPU device for training.
• not find GPU device for training.
• expected stages_repeats as list of 3 positive ints
• expected stages_out_channels as list of 5 positive ints

AI-assisted analysis of WZMIAOMIAO/deep-learning-for-image-processing@1ec3fe6f37 (2026-08-30). Data as JSON: /api/errors/2d51ad451c4158bb. Report an issue: GitHub.