{"record":{"id":"6eb9ed9bc0229035","repo":"sgl-project/sglang","slug":"unexpected-return-type-from-apply-chat-template","errorCode":null,"errorMessage":"Unexpected return type from apply_chat_template: {type(result)}","messagePattern":"Unexpected return type from apply_chat_template: (.+?)","errorType":"error_code","errorClass":"TypeError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/pipelines_core/stages/model_specific_stages/cosmos3.py","lineNumber":332,"sourceCode":"                    }\n                )\n            conversations.append({\"role\": \"user\", \"content\": text_item})\n\n            result = self.tokenizer.apply_chat_template(\n                conversations,\n                tokenize=True,\n                add_generation_prompt=True,\n            )\n            # Handle different return types from apply_chat_template\n            # Fast tokenizer returns BatchEncoding, slow tokenizer returns list[int]\n            if hasattr(result, \"input_ids\"):\n                # BatchEncoding from fast tokenizer\n                token_ids = list(result.input_ids)\n            elif isinstance(result, list):\n                # Already a list from slow tokenizer\n                token_ids = list(result)\n            else:\n                raise TypeError(\n                    f\"Unexpected return type from apply_chat_template: {type(result)}\"\n                )\n\n            # Reserve room for the two special tokens (EOS + vision_start) so the\n            # final length cannot exceed ``max_sequence_length``.\n            token_ids = token_ids[: max_sequence_length - 2]\n            # Add EOS and vision_start tokens\n            token_ids.append(self.tokenizer.eos_token_id)\n            if vision_start_id is not None:\n                token_ids.append(vision_start_id)\n\n            seq_len = len(token_ids)\n            pad_len = max_sequence_length - seq_len\n            attention_mask = [1] * seq_len + [0] * pad_len\n            token_ids = token_ids + [pad_token_id] * pad_len\n            input_id_lists.append(token_ids)\n            attention_mask_lists.append(attention_mask)\n            seq_lens.append(seq_len)","sourceCodeStart":314,"sourceCodeEnd":350,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/pipelines_core/stages/model_specific_stages/cosmos3.py#L314-L350","documentation":"After apply_chat_template(add_special_tokens=False, tokenize=True), the code accepts only a BatchEncoding (fast tokenizer, .input_ids) or a plain list (slow tokenizer). Any other return type — usually a str when tokenize=False, or a dict variant without input_ids — is a contract violation and raises TypeError.","triggerScenarios":"tokenize=True is expected but the tokenizer returns a str — this happens when apply_chat_template is called with tokenize=False semantics, or a transformers version changes the return shape; also exotic tokenizer subclasses returning dict/DataFrame-like objects.","commonSituations":"Upgrading/downgrading transformers where apply_chat_template's return type or kwargs behavior changed; a custom Qwen2 tokenizer subclass overriding apply_chat_template; accidentally passing tokenize=False through a wrapper.","solutions":["Pin/upgrade transformers to a version tested with this stage (where apply_chat_template(tokenize=True) returns BatchEncoding)","If using a custom tokenizer subclass, ensure its apply_chat_template returns input ids (BatchEncoding or list) when tokenize=True","Normalize defensively: call tokenizer.apply_chat_template(..., tokenize=True) yourself and pass ids, or coerce str via tokenizer(str) before the stage"],"exampleFix":"# before\nstage = Cosmos3TokenizationStage(tokenizer=custom_tok)  # custom_tok returns str\n# after\nids = custom_tok.apply_chat_template(msgs, add_special_tokens=False, tokenize=True)\nassert not isinstance(ids, str)","handlingStrategy":"type-guard","validationCode":"res = tok.apply_chat_template(msgs, add_special_tokens=False, tokenize=True)\nassert not isinstance(res, str) and (hasattr(res, \"input_ids\") or isinstance(res, list))","typeGuard":"def valid_template_result(res) -> bool:\n    return hasattr(res, \"input_ids\") or isinstance(res, list)","tryCatchPattern":"catch TypeError on 'Unexpected return type from apply_chat_template' and fall back to tokenizer(tok.apply_chat_template(msgs, tokenize=False))","preventionTips":["Pin the transformers version tested with the stage","Never call apply_chat_template with tokenize=False expecting ids"],"tags":["cosmos3","tokenizer","apply-chat-template","transformers","type-mismatch"],"backgroundTag":"unexpected-return-type","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}