{"record":{"id":"c00f314bbe2701d5","repo":"sgl-project/sglang","slug":"you-have-passed-a-list-of-generators-of-length-le","errorCode":null,"errorMessage":"You have passed a list of generators of length {len(generator)}, but requested an effective batch size of {batch_size}. Make sure the batch size matches the length of the generators.","messagePattern":"You have passed a list of generators of length (.+?), but requested an effective batch size of (.+?)\\. Make sure the batch size matches the length of the generators\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/pipelines_core/stages/latent_preparation.py","lineNumber":139,"sourceCode":"\n        batch_size = batch.batch_size\n\n        # Get required parameters\n        device = get_local_torch_device()\n        generator = batch.generator\n        latents = batch.latents\n        height = batch.height\n        width = batch.width\n\n        # TODO(will): remove this once we add input/output validation for stages\n        if self.requires_batch_height_width(batch, server_args) and (\n            height is None or width is None\n        ):\n            raise ValueError(\"Height and width must be provided\")\n\n        # Validate generator if it's a list\n        if isinstance(generator, list) and len(generator) != batch_size:\n            raise ValueError(\n                f\"You have passed a list of generators of length {len(generator)}, but requested an effective batch\"\n                f\" size of {batch_size}. Make sure the batch size matches the length of the generators.\"\n            )\n\n        # Generate or use provided latents\n        if latents is None:\n            spec = self.get_latent_preparation_spec(\n                batch, server_args, batch_size, latent_num_frames, device\n            )\n            latents = randn_tensor(\n                spec.shape,\n                generator=generator,\n                device=spec.device,\n                dtype=spec.dtype,\n            )\n\n            latent_ids = (\n                server_args.pipeline_config.maybe_prepare_latent_ids(latents)","sourceCodeStart":121,"sourceCodeEnd":157,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/pipelines_core/stages/latent_preparation.py#L121-L157","documentation":"When latents are seeded per-sample, a list of torch.Generator objects must match the effective batch size exactly. The stage validates len(generator) == batch_size (which can be larger than the prompt count when CFG doubles the batch) and rejects a mismatched list before sampling noise.","triggerScenarios":"Passing generator=[g1, g2] for a single prompt with CFG enabled (effective batch size 2x), or passing N generators for a different number of prompts; also passing a list where batch_size reflects num_prompts * cfg_multiplier.","commonSituations":"Reproducing diffusers seeds in a client script; enabling/disabling classifier-free guidance without resizing the generator list; batch-size changes from server-side batching while the generator list stays fixed.","solutions":["Match the generator list length to the effective (CFG-doubled) batch size, e.g. [gen] * batch_size","Or pass a single generator (not a list) and let the stage broadcast it","If deterministic output is all you need, pass explicit latents instead of generators"],"exampleFix":"# before\npipe(prompt, generator=[torch.Generator().manual_seed(0)])  # batch_size=2 under CFG\n# after\ngens = [torch.Generator().manual_seed(0) for _ in range(batch_size)]\npipe(prompt, generator=gens)","handlingStrategy":"validation","validationCode":"if isinstance(generator, list):\n    assert len(generator) == effective_batch_size, (len(generator), effective_batch_size)","typeGuard":"def valid_generator_arg(generator, batch_size) -> bool:\n    return generator is None or (not isinstance(generator, list)) or len(generator) == batch_size","tryCatchPattern":null,"preventionTips":["Remember CFG doubles the effective batch size","Prefer a single generator and let the stage broadcast it"],"tags":["generator","batch-size","seed","latent-preparation"],"backgroundTag":"batch-size-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}