sgl-project/sglang · error · ValueError

Generator must be provided

Error message

Generator must be provided

What it means

In the image-encoding stage's VAE latent sampling path, the per-batch random generator (batch.generator) is required to sample latents from the encoded distribution deterministically. If the batch was constructed without a torch.Generator (generator=None), the stage refuses to sample because reproducibility would be lost and downstream code assumes a per-request generator exists.

Source

Thrown at python/sglang/multimodal_gen/runtime/pipelines_core/stages/image_encoding.py:971

                    should_cast_vae = not vae_autocast_enabled
                    if not vae_autocast_enabled:
                        video_condition = video_condition.to(vae_dtype)
                    video_condition = server_args.pipeline_config.preprocess_vae_encode(
                        video_condition, self.vae
                    )
                    with temporary_module_dtype(
                        self.vae, vae_dtype, enabled=should_cast_vae
                    ) as vae:
                        latent_dist: DiagonalGaussianDistribution = vae.encode(
                            video_condition
                        )
                    # for auto_encoder from diffusers
                    if isinstance(latent_dist, AutoencoderKLOutput):
                        latent_dist = latent_dist.latent_dist

                generator = batch.generator
                if generator is None:
                    raise ValueError("Generator must be provided")

                sample_mode = (
                    server_args.pipeline_config.vae_config.encode_sample_mode()
                )

                latent_condition = self.retrieve_latents(
                    latent_dist, generator, sample_mode=sample_mode
                )
                latent_condition = server_args.pipeline_config.postprocess_vae_encode(
                    latent_condition, self.vae
                )
                normalized_latent_condition = (
                    server_args.pipeline_config.normalize_vae_encode(
                        latent_condition, self.vae
                    )
                )
                if normalized_latent_condition is None:
                    scaling_factor, shift_factor = (

View on GitHub (pinned to 0132848349)

Solutions

1. Set batch.generator = torch.Generator(device=...).manual_seed(seed) before submitting the request
2. If your server args accept a seed, verify the seed-handling path populates batch.generator rather than only a global seed
3. When constructing batches manually, mirror how the standard request path creates generators (device-matched, seeded per prompt)

Example fix

// before
batch = ImageBatch(images=imgs)  # generator missing

// after
gen = torch.Generator(device=latents_device).manual_seed(seed)
batch = ImageBatch(images=imgs, generator=gen)

if batch.generator is None:
    batch.generator = torch.Generator(device=device).manual_seed(seed or 0)

def has_generator(batch) -> bool:
    return getattr(batch, "generator", None) is not None

When it happens

Trigger: Building a request/batch object manually without setting generator, or a server path that only sets a global seed and forgets to attach a per-batch torch.Generator before reaching image encoding with vae_config.encode_sample_mode() requiring sampling.

Common situations: Calling the pipeline programmatically with a custom Batch dataclass; refactoring that dropped generator propagation; requests that set seed=0/None and a code path that then leaves batch.generator unset; version changes that made the generator mandatory for VAE sampling.

Related errors

• You have passed a list of generators of length {len(generato
• You have passed a list of generators of length {len(generato
• You have passed a list of generators of length {len(generato
• You have passed a list of generators of length {len(generato
• SANA-WM generator list must not be empty.

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/66fe3d715459a6ed. Report an issue: GitHub.