sgl-project/sglang · error · ValueError

You have passed a list of generators of length {len(generato

Error message

You have passed a list of generators of length {len(generator)}, but requested an effective batch size of {batch_size}. Make sure the batch size matches the length of the generators.

What it means

prepare_latents validates per-sample noise generators: if generator is a list, its length must equal the effective batch size so each sample gets reproducible, independent noise. A length mismatch makes seed-to-sample correspondence undefined, so it raises before sampling.

Source

Thrown at python/sglang/multimodal_gen/runtime/pipelines_core/stages/model_specific_stages/qwen_image_layered.py:465

                )
            else:
                image_latents = torch.cat([image_latents], dim=0)

            image_latent_height, image_latent_width = image_latents.shape[3:]
            image_latents = image_latents.permute(
                0, 2, 1, 3, 4
            )  # (b, c, f, h, w) -> (b, f, c, h, w)
            image_latents = self._pack_latents(
                image_latents,
                batch_size,
                num_channels_latents,
                image_latent_height,
                image_latent_width,
                1,
            )

        if isinstance(generator, list) and len(generator) != batch_size:
            raise ValueError(
                f"You have passed a list of generators of length {len(generator)}, but requested an effective batch"
                f" size of {batch_size}. Make sure the batch size matches the length of the generators."
            )
        if latents is None:
            latents = randn_tensor(
                shape, generator=generator, device=device, dtype=dtype
            )
            latents = self._pack_latents(
                latents, batch_size, num_channels_latents, height, width, layers + 1
            )
        else:
            latents = latents.to(device=device, dtype=dtype)

        return latents, image_latents

    def forward(
        self,
        batch: Req,

View on GitHub (pinned to 0132848349)

Solutions

Pass exactly batch_size generators: [torch.Generator(device).manual_seed(s) for s in range(batch_size)].
Or pass a single (non-list) generator to let one seed drive the whole batch.
Compute batch_size (prompts x variants) first, then build the generator list to match.

Example fix

# before
stage(..., prompts=["a", "b", "c"], generator=[g1, g2])

# after
gens = [torch.Generator(device="cuda").manual_seed(i) for i in range(3)]
stage(..., prompts=["a", "b", "c"], generator=gens)

Defensive patterns

Strategy: validation

Validate before calling

if isinstance(generator, list):
    assert len(generator) == batch_size, f"{len(generator)} generators vs batch {batch_size}"

Type guard

def generators_match_batch(generator, batch_size: int) -> bool:
    return not isinstance(generator, list) or len(generator) == batch_size

Prevention

Build the generator list from computed batch_size at call time.
Pass a single generator when per-sample seeds are not needed.

When it happens

Trigger: Passing generator=[torch.Generator(), torch.Generator()] (len 2) while batch_size is 1, 3, 4, ... — any combination where len(generator) != batch_size after prompt expansion/latent duplication determines batch_size.

Common situations: Hardcoding a fixed generator list while varying num_prompts; forgetting that prompt expansion multiplies the effective batch; refactoring from single generator to list without updating count; diffusers-style reproducibility code copied with wrong count.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/f2c2628769de6830. Report an issue: GitHub.