{"record":{"id":"6bdf6ac8eb96c946","repo":"CorentinJ/Real-Time-Voice-Cloning","slug":"batch-size-must-be-evenly-divisible-by-n-gpus","errorCode":null,"errorMessage":"`batch_size` must be evenly divisible by n_gpus!","messagePattern":"`batch_size` must be evenly divisible by n_gpus!","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"synthesizer/train.py","lineNumber":60,"sourceCode":"    weights_fpath = model_dir / f\"synthesizer.pt\"\n    metadata_fpath = syn_dir.joinpath(\"train.txt\")\n\n    print(\"Checkpoint path: {}\".format(weights_fpath))\n    print(\"Loading training data from: {}\".format(metadata_fpath))\n    print(\"Using model: Tacotron\")\n\n    # Bookkeeping\n    time_window = ValueWindow(100)\n    loss_window = ValueWindow(100)\n\n    # From WaveRNN/train_tacotron.py\n    if torch.cuda.is_available():\n        device = torch.device(\"cuda\")\n\n        for session in hparams.tts_schedule:\n            _, _, _, batch_size = session\n            if batch_size % torch.cuda.device_count() != 0:\n                raise ValueError(\"`batch_size` must be evenly divisible by n_gpus!\")\n    else:\n        device = torch.device(\"cpu\")\n    print(\"Using device:\", device)\n\n    # Instantiate Tacotron Model\n    print(\"\\nInitialising Tacotron Model...\\n\")\n    model = Tacotron(embed_dims=hparams.tts_embed_dims,\n                     num_chars=len(symbols),\n                     encoder_dims=hparams.tts_encoder_dims,\n                     decoder_dims=hparams.tts_decoder_dims,\n                     n_mels=hparams.num_mels,\n                     fft_bins=hparams.num_mels,\n                     postnet_dims=hparams.tts_postnet_dims,\n                     encoder_K=hparams.tts_encoder_K,\n                     lstm_dims=hparams.tts_lstm_dims,\n                     postnet_K=hparams.tts_postnet_K,\n                     num_highways=hparams.tts_num_highways,\n                     dropout=hparams.tts_dropout,","sourceCodeStart":42,"sourceCodeEnd":78,"githubUrl":"https://github.com/CorentinJ/Real-Time-Voice-Cloning/blob/890f3a03187195b9829db2079b75c2ba2ab0405c/synthesizer/train.py#L42-L78","documentation":"Raised by the training loop setup in synthesizer/train.py when CUDA is available and any batch_size entry in hparams.tts_schedule is not divisible by torch.cuda.device_count(). tts_schedule is a list of (lr, iters, clip, batch_size) annealing phases; every phase's batch must shard evenly across GPUs under DataParallel, so each entry is validated before training starts. Like error 6, the check only runs on CUDA machines.","triggerScenarios":"Running synthesizer_train.py on a multi-GPU machine with a tts_schedule containing at least one batch_size that is not a multiple of the visible GPU count (e.g. schedule [(1e-3, 100000, 1e-5, 11), ...] with 2 GPUs).","commonSituations":"Using a shared/rented multi-GPU box with a schedule authored for single GPU; changing CUDA_VISIBLE_DEVICES or moving to a different node with more GPUs; hand-editing the schedule and leaving one phase odd.","solutions":["Edit the tts_schedule in the synthesizer hparams so every 4th tuple element is a multiple of the GPU count (e.g. 16, 32, 48 on 2/4/8 GPUs).","Or pin the run to one GPU: CUDA_VISIBLE_DEVICES=0 python synthesizer_train.py <root>.","Print torch.cuda.device_count() in the same environment to confirm how many GPUs the schedule must divide into before editing."],"exampleFix":"# before: 2 GPUs\nhparams.tts_schedule = [(1e-3, 100000, 1e-5, 11), (5e-4, 100000, 1e-5, 11)]  # 11 % 2 -> ValueError\n\n# after\nhparams.tts_schedule = [(1e-3, 100000, 1e-5, 12), (5e-4, 100000, 1e-5, 12)]  # divisible by 2 (and 3, 4, 6)","handlingStrategy":"validation","validationCode":"import torch\n\ndef validate_schedule(schedule):\n    n = torch.cuda.device_count() if torch.cuda.is_available() else 1\n    for i, (_, _, _, bs) in enumerate(schedule):\n        if n > 1 and bs % n != 0:\n            raise ValueError(f\"tts_schedule[{i}].batch_size={bs} not divisible by {n} GPUs\")\n    return schedule","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Validate every phase of tts_schedule against the target machine's GPU count before submitting a training job.","Keep per-machine hparams overrides rather than editing the shared defaults.","Prefer powers-of-two batch sizes across the whole schedule."],"tags":["gpu","multi-gpu","synthesizer","hparams","training"],"backgroundTag":null,"analyzedSha":"890f3a03187195b9829db2079b75c2ba2ab0405c","analyzedAt":"2026-08-15T02:15:13.202Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}