sgl-project/sglang · error · ValueError

You have passed a list of generators of length {len(generato

Error message

You have passed a list of generators of length {len(generator)}, but requested an effective batch size of {batch_size}. Make sure the batch size matches the length of the generators.

What it means

MOVA latents-preparation stage validates that when the caller passes a list of random generators, its length must equal the effective batch size. Diffusers-style pipelines require one generator per sample (or a single shared generator); a length mismatch would silently produce wrong latent noise shaping, so it is rejected up front.

Source

Thrown at python/sglang/multimodal_gen/runtime/pipelines_core/stages/model_specific_stages/mova.py:108

    def forward(self, batch: Req, server_args: ServerArgs) -> Req:
        batch_size = batch.batch_size
        num_frames = batch.num_frames
        if num_frames is None:
            raise ValueError("num_frames is required for MOVA")

        audio_num_samples = int(self.audio_vae.sample_rate * num_frames / batch.fps)

        video_shape = server_args.pipeline_config.prepare_latent_shape(
            batch, batch_size, num_frames
        )
        audio_shape = server_args.pipeline_config.prepare_audio_latent_shape(
            batch_size, audio_num_samples, self.audio_vae
        )

        device = get_local_torch_device()
        generator = batch.generator
        if isinstance(generator, list) and len(generator) != batch_size:
            raise ValueError(
                f"You have passed a list of generators of length {len(generator)}, but requested an effective batch"
                f" size of {batch_size}. Make sure the batch size matches the length of the generators."
            )

        dit_dtype = PRECISION_TO_TYPE[server_args.pipeline_config.dit_precision]
        batch.latents = randn_tensor(
            video_shape, generator=generator, device=device, dtype=dit_dtype
        )
        batch.audio_latents = randn_tensor(
            audio_shape, generator=generator, device=device, dtype=dit_dtype
        )

        if batch.image_latent is not None:
            batch.y = batch.image_latent.to(device=device, dtype=dit_dtype)
        elif self.require_vae_embedding:
            raise ValueError("MOVA requires reference image latents for denoising")
        return batch

View on GitHub (pinned to 0132848349)

Solutions

Make len(generator) == batch_size: pass exactly one generator per sample in the batch
Or pass a single (non-list) generator object to share across the whole batch
If dynamic batching changed batch_size after generators were built, rebuild the generator list per batch

Example fix

# before
generators = [torch.Generator(device='cuda').manual_seed(seed)]
# after
generators = [torch.Generator(device='cuda').manual_seed(seed + i) for i in range(batch_size)]

Defensive patterns

Strategy: validation

Validate before calling

gens = request.generators
if isinstance(gens, list):
    assert len(gens) == batch_size, f'need {batch_size} generators, got {len(gens)}'
# or pass a single generator to share across the batch

Type guard

def is_valid_generator_arg(g, batch_size: int) -> bool:
    return (hasattr(g, 'device') and not isinstance(g, list)) or (
        isinstance(g, list) and len(g) == batch_size
        and all(hasattr(x, 'device') for x in g)
    )

Prevention

Build generator lists with a list comprehension sized by the computed batch size
Never hardcode [generator] when batch size is dynamic
Prefer a single shared generator unless per-sample seeds are required

When it happens

Trigger: Calling the MOVA pipeline forward with batch.generator being a Python list whose len() differs from the computed batch_size (derived from audio_num_samples and self.audio_vae). Common when batching requests while supplying per-request generators.

Common situations: Passing [torch.Generator()] * 1 for a batch of N requests, mixing a single-generator call pattern into a batched scheduler loop, or off-by-one when slicing generators to match a dynamic batch size.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/43aae69e30b2ee03. Report an issue: GitHub.