{"record":{"id":"e29e82fddf388731","repo":"hiyouga/LlamaFactory","slug":"vllm-engine-does-not-support-get-scores","errorCode":null,"errorMessage":"vLLM engine does not support `get_scores`.","messagePattern":"vLLM engine does not support `get_scores`\\.","errorType":"exception","errorClass":"NotImplementedError","httpStatus":null,"severity":"error","filePath":"src/llamafactory/chat/vllm_engine.py","lineNumber":273,"sourceCode":"        images: Optional[list[\"ImageInput\"]] = None,\n        videos: Optional[list[\"VideoInput\"]] = None,\n        audios: Optional[list[\"AudioInput\"]] = None,\n        **input_kwargs,\n    ) -> AsyncGenerator[str, None]:\n        generated_text = \"\"\n        generator = await self._generate(messages, system, tools, images, videos, audios, **input_kwargs)\n        async for result in generator:\n            delta_text = result.outputs[0].text[len(generated_text) :]\n            generated_text = result.outputs[0].text\n            yield delta_text\n\n    @override\n    async def get_scores(\n        self,\n        batch_input: list[str],\n        **input_kwargs,\n    ) -> list[float]:\n        raise NotImplementedError(\"vLLM engine does not support `get_scores`.\")\n","sourceCodeStart":255,"sourceCodeEnd":274,"githubUrl":"https://github.com/hiyouga/LlamaFactory/blob/f28afaf6355af515454dfb16c97d728307c93897/src/llamafactory/chat/vllm_engine.py#L255-L274","documentation":"vLLMEngine.get_scores unconditionally raises NotImplementedError. The vLLM chat backend in this codebase only exposes generation (chat/stream_chat); the sequence-scoring interface used for reward-model evaluation is not wired to vLLM's log-prob APIs, so any get_scores call fails fast.","triggerScenarios":"Calling await engine.get_scores(batch_input) on a vLLMEngine instance — e.g. an RM evaluation or preference-scoring script whose engine backend was configured as vllm.","commonSituations":"Switching inference_backend to vllm for speed and forgetting that the scoring path still needs the hf engine; shared evaluation code that assumes every engine implements get_scores.","solutions":["Run reward scoring with the hf engine instead of vllm.","Guard call sites with an engine-capability check so scoring code never dispatches to vLLMEngine."],"exampleFix":"# before\nscores = await engine.get_scores(batch_input)  # engine is vLLMEngine\n\n# after\nif type(engine).__name__ == \"vLLMEngine\":\n    raise SystemExit(\"scoring requires the hf engine\")\nscores = await engine.get_scores(batch_input)","handlingStrategy":"type-guard","validationCode":"assert type(engine).__name__ != \"vLLMEngine\", \"vllm engine cannot score sequences\"","typeGuard":"def supports_scoring(engine) -> bool:\n    return not hasattr(engine, \"_generate\") or type(engine).__name__ not in {\"vLLMEngine\", \"SGLangEngine\"}","tryCatchPattern":null,"preventionTips":["Route RM evaluation jobs to the hf engine by policy.","Add an integration test asserting your scoring entrypoint rejects non-scoring backends early."],"tags":["vllm","not-implemented","reward-model","inference"],"backgroundTag":null,"analyzedSha":"f28afaf6355af515454dfb16c97d728307c93897","analyzedAt":"2026-08-14T21:57:28.298Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}