sgl-project/sglang · error · ValueError

Cosmos3 prompt batch must not be empty

Error message

Cosmos3 prompt batch must not be empty

What it means

_tokenize_prompt normalizes its input to a list of texts and requires at least one entry. An empty list (or empty tuple) means there are no prompts to tokenize, so the stage raises instead of producing an empty batch.

Source

Thrown at python/sglang/multimodal_gen/runtime/pipelines_core/stages/model_specific_stages/cosmos3.py:301

        result = VerificationResult()
        result.add_check("prompt", batch.prompt, V.string_or_list_strings)
        return result

    def _tokenize_prompt(
        self,
        text: str | list[str],
        max_sequence_length: int,
        device: torch.device,
        use_system_prompt: bool = False,
        system_prompt: str | None = None,
    ) -> tuple[torch.Tensor, torch.Tensor, int]:
        """Tokenize a prompt using Qwen2 chat template.

        Returns (input_ids, attention_mask, seq_len) as [B, S] tensors.
        """
        texts = text if isinstance(text, (list, tuple)) else [text]
        if not texts:
            raise ValueError("Cosmos3 prompt batch must not be empty")
        input_id_lists: list[list[int]] = []
        attention_mask_lists: list[list[int]] = []
        seq_lens: list[int] = []
        pad_token_id = self.tokenizer.pad_token_id or 0
        vision_start_id = self.tokenizer.convert_tokens_to_ids("<|vision_start|>")
        for text_item in texts:
            conversations = []
            if use_system_prompt:
                conversations.append(
                    {
                        "role": "system",
                        "content": system_prompt or COSMOS3_VIDEO_SYSTEM_PROMPT,
                    }
                )
            conversations.append({"role": "user", "content": text_item})

            result = self.tokenizer.apply_chat_template(
                conversations,

View on GitHub (pinned to 0132848349)

Solutions

1. Skip the call entirely when the prompt list is empty (guard at the caller)
2. Ensure upstream filtering/dequeuing never forwards an empty batch downstream
3. Pass at least one prompt string

Example fix

# before
ids, mask, lens = stage._tokenize_prompt([], tokenizer_max_length)
# after
if not prompts:
    return  # or continue
ids, mask, lens = stage._tokenize_prompt(prompts, tokenizer_max_length)

prompts = list(prompts) if isinstance(prompts, (list, tuple)) else [prompts]
if not prompts:
    return  # skip empty batch

def nonempty_prompt_batch(text) -> bool:
    texts = list(text) if isinstance(text, (list, tuple)) else [text]
    return len(texts) > 0

When it happens

Trigger: Calling the pipeline/stage with prompt=[] (or the tokenization helper directly with an empty list) — e.g. a batching layer that flushed an empty micro-batch, or upstream filtering removed all prompts.

Common situations: Dynamic batching where a filter (safety, length) removes every prompt; a UI submitting an empty prompt list; tests iterating over an empty dataset.

Related errors

• Cosmos3 batched prompts must tokenize to the same length bec
• Prompt cannot be empty
• Cosmos3 action input accepts one image field; use a list or
• Cosmos3 observation image arrays must use uint8 dtype
• Cosmos3 observation image arrays must have shape [H, W] or [

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/3f4c0fda07a297be. Report an issue: GitHub.