sgl-project/sglang · error · TypeError

Unexpected return type from apply_chat_template: {type(resul

Error message

Unexpected return type from apply_chat_template: {type(result)}

What it means

After apply_chat_template(add_special_tokens=False, tokenize=True), the code accepts only a BatchEncoding (fast tokenizer, .input_ids) or a plain list (slow tokenizer). Any other return type — usually a str when tokenize=False, or a dict variant without input_ids — is a contract violation and raises TypeError.

Source

Thrown at python/sglang/multimodal_gen/runtime/pipelines_core/stages/model_specific_stages/cosmos3.py:332

                    }
                )
            conversations.append({"role": "user", "content": text_item})

            result = self.tokenizer.apply_chat_template(
                conversations,
                tokenize=True,
                add_generation_prompt=True,
            )
            # Handle different return types from apply_chat_template
            # Fast tokenizer returns BatchEncoding, slow tokenizer returns list[int]
            if hasattr(result, "input_ids"):
                # BatchEncoding from fast tokenizer
                token_ids = list(result.input_ids)
            elif isinstance(result, list):
                # Already a list from slow tokenizer
                token_ids = list(result)
            else:
                raise TypeError(
                    f"Unexpected return type from apply_chat_template: {type(result)}"
                )

            # Reserve room for the two special tokens (EOS + vision_start) so the
            # final length cannot exceed ``max_sequence_length``.
            token_ids = token_ids[: max_sequence_length - 2]
            # Add EOS and vision_start tokens
            token_ids.append(self.tokenizer.eos_token_id)
            if vision_start_id is not None:
                token_ids.append(vision_start_id)

            seq_len = len(token_ids)
            pad_len = max_sequence_length - seq_len
            attention_mask = [1] * seq_len + [0] * pad_len
            token_ids = token_ids + [pad_token_id] * pad_len
            input_id_lists.append(token_ids)
            attention_mask_lists.append(attention_mask)
            seq_lens.append(seq_len)

View on GitHub (pinned to 0132848349)

Solutions

1. Pin/upgrade transformers to a version tested with this stage (where apply_chat_template(tokenize=True) returns BatchEncoding)
2. If using a custom tokenizer subclass, ensure its apply_chat_template returns input ids (BatchEncoding or list) when tokenize=True
3. Normalize defensively: call tokenizer.apply_chat_template(..., tokenize=True) yourself and pass ids, or coerce str via tokenizer(str) before the stage

Example fix

# before
stage = Cosmos3TokenizationStage(tokenizer=custom_tok)  # custom_tok returns str
# after
ids = custom_tok.apply_chat_template(msgs, add_special_tokens=False, tokenize=True)
assert not isinstance(ids, str)

res = tok.apply_chat_template(msgs, add_special_tokens=False, tokenize=True)
assert not isinstance(res, str) and (hasattr(res, "input_ids") or isinstance(res, list))

def valid_template_result(res) -> bool:
    return hasattr(res, "input_ids") or isinstance(res, list)

catch TypeError on 'Unexpected return type from apply_chat_template' and fall back to tokenizer(tok.apply_chat_template(msgs, tokenize=False))

When it happens

Trigger: tokenize=True is expected but the tokenizer returns a str — this happens when apply_chat_template is called with tokenize=False semantics, or a transformers version changes the return shape; also exotic tokenizer subclasses returning dict/DataFrame-like objects.

Common situations: Upgrading/downgrading transformers where apply_chat_template's return type or kwargs behavior changed; a custom Qwen2 tokenizer subclass overriding apply_chat_template; accidentally passing tokenize=False through a wrapper.

Related errors

• Cosmos3TokenizationStage requires a tokenizer; expected the
• Retry with use_fast=False for {tokenizer_name} also failed (
• Unsupported type {type(data)}
• All tensors must have the same data type
• Cosmos3 action input accepts one image field; use a list or

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/6eb9ed9bc0229035. Report an issue: GitHub.