{"record":{"id":"e4a009a1bca32a1e","repo":"hiyouga/LlamaFactory","slug":"cannot-get-scores-using-an-auto-regressive-model","errorCode":null,"errorMessage":"Cannot get scores using an auto-regressive model.","messagePattern":"Cannot get scores using an auto-regressive model\\.","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/llamafactory/chat/hf_engine.py","lineNumber":419,"sourceCode":"            audios,\n            input_kwargs,\n        )\n        async with self.semaphore:\n            stream = self._stream_chat(*input_args)\n            while True:\n                try:\n                    yield await asyncio.to_thread(stream)\n                except StopAsyncIteration:\n                    break\n\n    @override\n    async def get_scores(\n        self,\n        batch_input: list[str],\n        **input_kwargs,\n    ) -> list[float]:\n        if self.can_generate:\n            raise ValueError(\"Cannot get scores using an auto-regressive model.\")\n\n        input_args = (self.model, self.tokenizer, batch_input, input_kwargs)\n        async with self.semaphore:\n            return await asyncio.to_thread(self._get_scores, *input_args)\n","sourceCodeStart":401,"sourceCodeEnd":424,"githubUrl":"https://github.com/hiyouga/LlamaFactory/blob/f28afaf6355af515454dfb16c97d728307c93897/src/llamafactory/chat/hf_engine.py#L401-L424","documentation":"ValueError raised by HuggingfaceEngine.get_scores when self.can_generate is true — i.e. the loaded model is an auto-regressive (generative) causal LM, but scoring requires a sequence-classification/reward model. The engine enforces the inverse constraint of chat/stream_chat.","triggerScenarios":"Calling get_scores / the score-evaluation endpoint with a standard instruct LLM (e.g. Qwen2.5-Instruct) loaded on the HF backend; running reward-data evaluation against the base chat model instead of the trained RM.","commonSituations":"Forgetting to switch model_name_or_path to the trained reward model before scoring; assuming any model can produce log-prob-based scores via this API.","solutions":["Point model_name_or_path at a reward/sequence-classification checkpoint (trained with stage rm).","Use chat/stream_chat if you actually want generations from a causal LM.","Check config.json: ForSequenceClassification architecture indicates a scorer; ForCausalLM indicates a generator."],"exampleFix":"# before\nchat_model = ChatModel({'model_name_or_path': 'Qwen/Qwen2.5-7B-Instruct'})\nscores = await chat_model.aget_scores([...])\n# after\nchat_model = ChatModel({'model_name_or_path': 'outputs/rm_checkpoint'})\nscores = await chat_model.aget_scores([...])","handlingStrategy":"validation","validationCode":"import json\ndef is_scorer(model_dir):\n    archs = json.load(open(f\"{model_dir}/config.json\"))[\"architectures\"]\n    return any(\"SequenceClassification\" in a for a in archs)\n\nassert is_scorer(model_path)  # before calling get_scores","typeGuard":null,"tryCatchPattern":"try { scores = await chat_model.aget_scores(batch) } except ValueError as e: if 'auto-regressive' in str(e): raise SystemExit(f'{model_path} cannot score; load an RM checkpoint') from e else: raise","preventionTips":["Load the trained RM checkpoint (stage rm output) for scoring workflows.","Inspect architectures in config.json to classify checkpoints.","Keep generation and scoring model handles separate in your service."],"tags":["model-loading","hf-engine","scoring","reward-model"],"backgroundTag":null,"analyzedSha":"f28afaf6355af515454dfb16c97d728307c93897","analyzedAt":"2026-08-14T21:57:28.298Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}