sgl-project/sglang · error · ValueError

SANA-WM refiner requires a string prompt or one prompt per b

Error message

SANA-WM refiner requires a string prompt or one prompt per batch item.

What it means

The SANA-WM refiner stage resolves prompts per batch item via _prompts_for_batch. It accepts exactly three shapes: a single string (broadcast to the batch), a list of strings whose length equals batch_size, or a one-element list of strings (broadcast). Anything else — an int, None, a list of non-strings, or a list whose length is neither 1 nor batch_size — raises this ValueError.

Source

Thrown at python/sglang/multimodal_gen/runtime/pipelines_core/stages/model_specific_stages/sana_wm/refiner.py:479

                component_name="transformer_2",
                target_dtype=self.dtype,
                memory_intensive=True,
            ),
        ]

    @staticmethod
    def _prompts_for_batch(batch: Req, batch_size: int) -> list[str]:
        prompt = batch.extra.get("refiner_prompt") if batch.extra else None
        if prompt is None:
            prompt = batch.prompt
        if isinstance(prompt, str):
            return [prompt] * batch_size
        if isinstance(prompt, list) and all(isinstance(p, str) for p in prompt):
            if len(prompt) == batch_size:
                return prompt
            if len(prompt) == 1:
                return prompt * batch_size
        raise ValueError(
            "SANA-WM refiner requires a string prompt or one prompt per batch item."
        )

    @torch.inference_mode()
    def _encode_prompt(
        self,
        prompt: str,
        device: torch.device,
    ) -> tuple[torch.Tensor, torch.Tensor]:
        tokenizer = self.tokenizer
        if getattr(tokenizer, "padding_side", "right") != "left":
            tokenizer.padding_side = "left"
        if tokenizer.pad_token is None and tokenizer.eos_token is not None:
            tokenizer.pad_token = tokenizer.eos_token

        text_inputs = tokenizer(
            [prompt.strip()],
            padding="max_length",

View on GitHub (pinned to 0132848349)

Solutions

Pass a single string prompt and let it broadcast to the whole batch
Ensure the prompt list length matches batch_size exactly (or use a 1-element list)
Sanitize upstream: coerce non-string entries to str and re-derive the list after any batch resize

Example fix

// before
refiner.forward(batch, server_args)  # batch prompt list len 3, batch_size 2
// after
prompts = prompts if len(prompts) == batch_size else [prompts[0]] * batch_size
batch.prompt = prompts
refiner.forward(batch, server_args)

Defensive patterns

Strategy: validation

Validate before calling

def valid_prompts(prompt, batch_size):
    if isinstance(prompt, str):
        return [prompt] * batch_size
    if isinstance(prompt, list) and all(isinstance(p, str) for p in prompt) and len(prompt) in (1, batch_size):
        return prompt * batch_size if len(prompt) == 1 else prompt
    return None

prompts = valid_prompts(batch.prompt, batch_size)
assert prompts is not None, 'invalid prompt spec for refiner'

Type guard

def is_valid_refiner_prompt(p: object, batch_size: int) -> bool:
    if isinstance(p, str):
        return True
    return isinstance(p, list) and all(isinstance(x, str) for x in p) and len(p) in (1, batch_size)

Try / catch

try:
    out = refiner.forward(batch, server_args)
except ValueError as e:
    if "one prompt per batch item" in str(e):
        batch.prompt = str(batch.prompt)
        out = refiner.forward(batch, server_args)
    else:
        raise

Prevention

Derive prompt lists from the same batch construction that sets batch_size
Coerce prompt fields to str at the request boundary
Add a unit test mirroring test_prompt_resolution_accepts_batch_prompt_list for your custom paths

When it happens

Trigger: Calling forward() on the refiner stage with batch prompt metadata that is not a str, or a list of str with len != batch_size and len != 1 (e.g. a 3-prompt list for a 2-item batch, or a list containing None/tensors).

Common situations: Upstream stage produces one prompt-embedding per item but the refiner is fed a mismatched prompt list; batch size changes (chunking/merging of Reqs) after prompts were materialized; prompt defaults to None when not supplied in the request.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/307ed186db6c88f9. Report an issue: GitHub.