{"record":{"id":"43aae69e30b2ee03","repo":"sgl-project/sglang","slug":"you-have-passed-a-list-of-generators-of-length-le-43aae6","errorCode":null,"errorMessage":"You have passed a list of generators of length {len(generator)}, but requested an effective batch size of {batch_size}. Make sure the batch size matches the length of the generators.","messagePattern":"You have passed a list of generators of length (.+?), but requested an effective batch size of (.+?)\\. Make sure the batch size matches the length of the generators\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/pipelines_core/stages/model_specific_stages/mova.py","lineNumber":108,"sourceCode":"    def forward(self, batch: Req, server_args: ServerArgs) -> Req:\n        batch_size = batch.batch_size\n        num_frames = batch.num_frames\n        if num_frames is None:\n            raise ValueError(\"num_frames is required for MOVA\")\n\n        audio_num_samples = int(self.audio_vae.sample_rate * num_frames / batch.fps)\n\n        video_shape = server_args.pipeline_config.prepare_latent_shape(\n            batch, batch_size, num_frames\n        )\n        audio_shape = server_args.pipeline_config.prepare_audio_latent_shape(\n            batch_size, audio_num_samples, self.audio_vae\n        )\n\n        device = get_local_torch_device()\n        generator = batch.generator\n        if isinstance(generator, list) and len(generator) != batch_size:\n            raise ValueError(\n                f\"You have passed a list of generators of length {len(generator)}, but requested an effective batch\"\n                f\" size of {batch_size}. Make sure the batch size matches the length of the generators.\"\n            )\n\n        dit_dtype = PRECISION_TO_TYPE[server_args.pipeline_config.dit_precision]\n        batch.latents = randn_tensor(\n            video_shape, generator=generator, device=device, dtype=dit_dtype\n        )\n        batch.audio_latents = randn_tensor(\n            audio_shape, generator=generator, device=device, dtype=dit_dtype\n        )\n\n        if batch.image_latent is not None:\n            batch.y = batch.image_latent.to(device=device, dtype=dit_dtype)\n        elif self.require_vae_embedding:\n            raise ValueError(\"MOVA requires reference image latents for denoising\")\n        return batch\n","sourceCodeStart":90,"sourceCodeEnd":126,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/pipelines_core/stages/model_specific_stages/mova.py#L90-L126","documentation":"MOVA latents-preparation stage validates that when the caller passes a list of random generators, its length must equal the effective batch size. Diffusers-style pipelines require one generator per sample (or a single shared generator); a length mismatch would silently produce wrong latent noise shaping, so it is rejected up front.","triggerScenarios":"Calling the MOVA pipeline forward with batch.generator being a Python list whose len() differs from the computed batch_size (derived from audio_num_samples and self.audio_vae). Common when batching requests while supplying per-request generators.","commonSituations":"Passing [torch.Generator()] * 1 for a batch of N requests, mixing a single-generator call pattern into a batched scheduler loop, or off-by-one when slicing generators to match a dynamic batch size.","solutions":["Make len(generator) == batch_size: pass exactly one generator per sample in the batch","Or pass a single (non-list) generator object to share across the whole batch","If dynamic batching changed batch_size after generators were built, rebuild the generator list per batch"],"exampleFix":"# before\ngenerators = [torch.Generator(device='cuda').manual_seed(seed)]\n# after\ngenerators = [torch.Generator(device='cuda').manual_seed(seed + i) for i in range(batch_size)]","handlingStrategy":"validation","validationCode":"gens = request.generators\nif isinstance(gens, list):\n    assert len(gens) == batch_size, f'need {batch_size} generators, got {len(gens)}'\n# or pass a single generator to share across the batch","typeGuard":"def is_valid_generator_arg(g, batch_size: int) -> bool:\n    return (hasattr(g, 'device') and not isinstance(g, list)) or (\n        isinstance(g, list) and len(g) == batch_size\n        and all(hasattr(x, 'device') for x in g)\n    )","tryCatchPattern":null,"preventionTips":["Build generator lists with a list comprehension sized by the computed batch size","Never hardcode [generator] when batch size is dynamic","Prefer a single shared generator unless per-sample seeds are required"],"tags":["mova","generator","batch-size","diffusers","validation"],"backgroundTag":"generator-batch-size-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}