sgl-project/sglang · error · ValueError

You have passed a list of generators of length {len(generato

Error message

You have passed a list of generators of length {len(generator)}, but requested an effective batch size of {batch_size}. Make sure the batch size matches the length of the generators.

What it means

prepare_latents validates that when generator is a list (per-sample generators for reproducible sampling), its length must equal the effective batch size of the request. A mismatch means some samples would have no defined generator, so the initial noise latents cannot be drawn deterministically.

Source

Thrown at python/sglang/multimodal_gen/runtime/pipelines_core/stages/model_specific_stages/glm_image.py:1066

    def prepare_latents(
        self,
        batch_size,
        num_channels_latents,
        height,
        width,
        dtype,
        device,
        generator,
    ):

        shape = (
            batch_size,
            num_channels_latents,
            int(height) // self.vae_scale_factor,
            int(width) // self.vae_scale_factor,
        )
        if isinstance(generator, list) and len(generator) != batch_size:
            raise ValueError(
                f"You have passed a list of generators of length {len(generator)}, but requested an effective batch"
                f" size of {batch_size}. Make sure the batch size matches the length of the generators."
            )
        latents = randn_tensor(shape, generator=generator, device=device, dtype=dtype)
        return latents

    def check_inputs(
        self,
        prompt,
        height,
        width,
        callback_on_step_end_tensor_inputs,
        prompt_embeds=None,
    ):
        if (
            height is not None
            and height % (self.vae_scale_factor * self.transformer.config.patch_size)
            != 0

View on GitHub (pinned to 0132848349)

Solutions

Pass a single torch.Generator instead of a list when per-sample control is not needed
Or build the list to match: generator=[torch.Generator(device).manual_seed(s) for s in range(batch_size)]
Account for num_images_per_prompt: effective batch = len(prompt) * num_images_per_prompt

Example fix

# before
pipe(prompt=["a", "b", "c"], generator=[g0, g1])

# after
gens = [torch.Generator("cuda").manual_seed(42 + i) for i in range(3)]
pipe(prompt=["a", "b", "c"], generator=gens)
# or simply: pipe(prompt=["a", "b", "c"], generator=torch.Generator("cuda").manual_seed(42))

Defensive patterns

Strategy: validation

Validate before calling

batch_size = len(prompt) * num_images_per_prompt
if isinstance(generator, list) and len(generator) != batch_size:
    generator = generator[:1] * batch_size  # or rebuild
# simplest: pass a single generator

Type guard

def generator_matches(generator, batch_size) -> bool:
    return not isinstance(generator, list) or len(generator) == batch_size

Try / catch

except ValueError as e:
    if "list of generators" in str(e):
        pipe(prompt=prompts, generator=generator[0])  # single generator fallback
    else:
        raise

Prevention

Default to a single torch.Generator unless per-sample seeds are needed
Derive generator list length from len(prompt) * num_images_per_prompt
Rebuild generator lists per request instead of reusing stale ones

When it happens

Trigger: Calling the pipeline with generator=[g1, g2] but a prompt list of length 3 (or a single prompt repeated into batch size 3 via num_images_per_prompt); any case where len(generator list) != computed batch_size of the latents shape.

Common situations: Setting num_images_per_prompt > 1 while passing one generator per prompt; batching prompts and reusing a stale generator list from a previous single-prompt run; torch.Generator lists built with a hard-coded length.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/363330257431d7e4. Report an issue: GitHub.