sgl-project/sglang · error · ValueError

You have passed a list of generators of length {len(generato

Error message

You have passed a list of generators of length {len(generator)}, but requested an effective batch size of {batch_size}. Make sure the batch size matches the length of the generators.

What it means

When latents are seeded per-sample, a list of torch.Generator objects must match the effective batch size exactly. The stage validates len(generator) == batch_size (which can be larger than the prompt count when CFG doubles the batch) and rejects a mismatched list before sampling noise.

Source

Thrown at python/sglang/multimodal_gen/runtime/pipelines_core/stages/latent_preparation.py:139

        batch_size = batch.batch_size

        # Get required parameters
        device = get_local_torch_device()
        generator = batch.generator
        latents = batch.latents
        height = batch.height
        width = batch.width

        # TODO(will): remove this once we add input/output validation for stages
        if self.requires_batch_height_width(batch, server_args) and (
            height is None or width is None
        ):
            raise ValueError("Height and width must be provided")

        # Validate generator if it's a list
        if isinstance(generator, list) and len(generator) != batch_size:
            raise ValueError(
                f"You have passed a list of generators of length {len(generator)}, but requested an effective batch"
                f" size of {batch_size}. Make sure the batch size matches the length of the generators."
            )

        # Generate or use provided latents
        if latents is None:
            spec = self.get_latent_preparation_spec(
                batch, server_args, batch_size, latent_num_frames, device
            )
            latents = randn_tensor(
                spec.shape,
                generator=generator,
                device=spec.device,
                dtype=spec.dtype,
            )

            latent_ids = (
                server_args.pipeline_config.maybe_prepare_latent_ids(latents)

View on GitHub (pinned to 0132848349)

Solutions

1. Match the generator list length to the effective (CFG-doubled) batch size, e.g. [gen] * batch_size
2. Or pass a single generator (not a list) and let the stage broadcast it
3. If deterministic output is all you need, pass explicit latents instead of generators

Example fix

# before
pipe(prompt, generator=[torch.Generator().manual_seed(0)])  # batch_size=2 under CFG
# after
gens = [torch.Generator().manual_seed(0) for _ in range(batch_size)]
pipe(prompt, generator=gens)

if isinstance(generator, list):
    assert len(generator) == effective_batch_size, (len(generator), effective_batch_size)

def valid_generator_arg(generator, batch_size) -> bool:
    return generator is None or (not isinstance(generator, list)) or len(generator) == batch_size

When it happens

Trigger: Passing generator=[g1, g2] for a single prompt with CFG enabled (effective batch size 2x), or passing N generators for a different number of prompts; also passing a list where batch_size reflects num_prompts * cfg_multiplier.

Common situations: Reproducing diffusers seeds in a client script; enabling/disabling classifier-free guidance without resizing the generator list; batch-size changes from server-side batching while the generator list stays fixed.

Related errors

• SANA-WM seed list length must be 1 or match latent batch siz
• You have passed a list of generators of length {len(generato
• You have passed a list of generators of length {len(generato
• You have passed a list of generators of length {len(generato
• SANA-WM generator list length must match latent batch size;

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/c00f314bbe2701d5. Report an issue: GitHub.