{"record":{"id":"71a650d486ad722e","repo":"hiyouga/LlamaFactory","slug":"the-current-model-does-not-support-chat","errorCode":null,"errorMessage":"The current model does not support `chat`.","messagePattern":"The current model does not support `chat`\\.","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/llamafactory/chat/hf_engine.py","lineNumber":357,"sourceCode":"            add_special_tokens=False,\n        ).to(device)\n        values: torch.Tensor = model(**inputs, return_dict=True, use_cache=False)[-1]\n        scores = values.gather(dim=-1, index=(inputs[\"attention_mask\"].sum(dim=-1, keepdim=True) - 1))\n        return scores\n\n    @override\n    async def chat(\n        self,\n        messages: list[dict[str, str]],\n        system: Optional[str] = None,\n        tools: Optional[str] = None,\n        images: Optional[list[\"ImageInput\"]] = None,\n        videos: Optional[list[\"VideoInput\"]] = None,\n        audios: Optional[list[\"AudioInput\"]] = None,\n        **input_kwargs,\n    ) -> list[\"Response\"]:\n        if not self.can_generate:\n            raise ValueError(\"The current model does not support `chat`.\")\n\n        input_args = (\n            self.model,\n            self.tokenizer,\n            self.processor,\n            self.template,\n            self.generating_args,\n            messages,\n            system,\n            tools,\n            images,\n            videos,\n            audios,\n            input_kwargs,\n        )\n        async with self.semaphore:\n            return await asyncio.to_thread(self._chat, *input_args)\n","sourceCodeStart":339,"sourceCodeEnd":375,"githubUrl":"https://github.com/hiyouga/LlamaFactory/blob/f28afaf6355af515454dfb16c97d728307c93897/src/llamafactory/chat/hf_engine.py#L339-L375","documentation":"ValueError raised by HuggingfaceEngine.chat when the loaded model cannot generate (can_generate is false — typically a sequence-classification/RM-style checkpoint). Chat is a generation task; the HF engine routes scoring-only models to get_scores instead.","triggerScenarios":"ChatModel(...) built with a reward/model classifier checkpoint (e.g. a merged reward model dir) and then calling .chat() or hitting /v1/chat/completions; loading an encoder-only or seq-cls model where AutoModelForCausalLM is not applicable.","commonSituations":"Reusing a fine-tuned reward-model output dir as a chat model; pointing model_name_or_path at a classifier; exporting then chatting with an RM checkpoint.","solutions":["Use a causal-LM checkpoint (AutoModelForCausalLM-compatible) for chat.","If the model is a scorer, call get_scores / the score-evaluation endpoint instead of chat.","Check config.json architectures in the model dir — ForSequenceClassification indicates a scorer.","Re-export or fine-tune with the correct stage (sft) to get a generative model."],"exampleFix":"# before\nchat_model = ChatModel({'model_name_or_path': 'outputs/reward_model_dir'})\nresp = chat_model.chat([...])\n# after\nchat_model = ChatModel({'model_name_or_path': 'qwen/Qwen2.5-7B-Instruct'})\nresp = chat_model.chat([...])","handlingStrategy":"validation","validationCode":"import json\ndef is_generative(model_dir):\n    archs = json.load(open(f\"{model_dir}/config.json\"))[\"architectures\"]\n    return any(\"CausalLM\" in a or \"LMHead\" in a for a in archs)\n\nassert is_generative(model_path)","typeGuard":null,"tryCatchPattern":"try { await chat_model.achat(msgs) } except ValueError as e: if 'does not support `chat`' in str(e): raise SystemExit(f'{model_path} is a scorer; use get_scores') from e else: raise","preventionTips":["Check config.json architectures before wiring a checkpoint into chat.","Keep chat checkpoints and reward/score checkpoints in clearly named dirs.","Prefer explicit stage labels in export metadata."],"tags":["model-loading","hf-engine","inference","model-type"],"backgroundTag":null,"analyzedSha":"f28afaf6355af515454dfb16c97d728307c93897","analyzedAt":"2026-08-14T21:57:28.298Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}