{"record":{"id":"ed03572dd325d1c1","repo":"vllm-project/vllm","slug":"image-generation-should-not-fail","errorCode":null,"errorMessage":"Image generation should not fail","messagePattern":"Image generation should not fail","errorType":"panic","errorClass":null,"httpStatus":null,"severity":"error","filePath":"rust/src/bench/src/datasets/random_mm.rs","lineNumber":443,"sourceCode":"            Ok(text)\n        })\n        .collect();\n    let prompts = prompts?;\n\n    // Generate images for each request (parallel per request)\n    // Each request gets its own RNG seeded deterministically.\n    let rid_prefix = request_id_prefix.to_string();\n    let result: Vec<SampleRequest> = prompts\n        .into_par_iter()\n        .enumerate()\n        .map(|(i, prompt)| {\n            let mut item_rng =\n                StdRng::seed_from_u64(seed.wrapping_add(i as u64).wrapping_add(0xBEEF));\n            let mm_items: Vec<Arc<str>> = mm_configs[i]\n                .iter()\n                .map(|key| {\n                    generate_random_image(key.width, key.height, &mut item_rng)\n                        .expect(\"Image generation should not fail\")\n                })\n                .collect();\n\n            let mm_content: Option<Arc<[Arc<str>]>> = if mm_items.is_empty() {\n                None\n            } else {\n                Some(Arc::from(mm_items))\n            };\n\n            // --enable-multimodal-chat: pre-build the full chat `messages` array\n            // (text part + mm items) at dataset time, mirroring Python's\n            // apply_multimodal_chat_transformation. mm content moves inside the\n            // messages string; the backend splices it verbatim.\n            let (mm_content, chat_messages_json) = if enable_multimodal_chat {\n                let msgs = build_chat_messages_json(&prompt, mm_content.as_deref());\n                (None, Some(Arc::from(msgs.as_str())))\n            } else {\n                (mm_content, None)","sourceCodeStart":425,"sourceCodeEnd":461,"githubUrl":"https://github.com/vllm-project/vllm/blob/c794754062d49a8fdb63ab3c5215b488b865030c/rust/src/bench/src/datasets/random_mm.rs#L425-L461","documentation":"The PyNCCL EPLB communicator uses NCCL collectives for expert-weight redistribution, which only work on CUDA-like (GPU) tensors. The expert weight tensors are on CPU, so this backend cannot be constructed.","triggerScenarios":"Requesting backend='pynccl' when expert weights live on CPU, e.g. CPU offloading of MoE weights or a stateless/meta-device initialization phase where tensors are still on CPU.","commonSituations":"CPU MoE inference setups; initialization order where the communicator factory runs before weights are moved to GPU; test harnesses constructing the communicator with CPU tensors.","solutions":["Use the 'torch_gloo' backend for CPU expert weights (it is built on the CPU gloo group)","Move expert weights to GPU before creating the pynccl EPLB communicator"],"exampleFix":"# before\ncomm = create_eplb_communicator(..., backend=\"pynccl\")  # weights on CPU\n\n# after\ncomm = create_eplb_communicator(..., backend=\"torch_gloo\")","handlingStrategy":"type-guard","validationCode":"device_type = expert_weights[0][0].device.type if expert_weights and expert_weights[0] else 'cpu'\nbackend = 'pynccl' if device_type == 'cuda' else 'torch_gloo'","typeGuard":"def weights_on_cuda_like(expert_weights) -> bool:\n    return bool(expert_weights) and expert_weights[0][0].device.type.startswith('cuda')","tryCatchPattern":null,"preventionTips":["Select the EPLB backend from the actual tensor device, not a hardcoded string","Create the communicator only after weights are resident on the target device"],"tags":["eplb","pynccl","nccl","device-mismatch","vllm"],"backgroundTag":null,"analyzedSha":"c794754062d49a8fdb63ab3c5215b488b865030c","analyzedAt":"2026-08-14T21:17:39.825Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}