{"record":{"id":"6ebd52aee1985f90","repo":"hiyouga/LlamaFactory","slug":"sglang-engine-does-not-support-get-scores","errorCode":null,"errorMessage":"SGLang engine does not support `get_scores`.","messagePattern":"SGLang engine does not support `get_scores`\\.","errorType":"exception","errorClass":"NotImplementedError","httpStatus":null,"severity":"error","filePath":"src/llamafactory/chat/sglang_engine.py","lineNumber":284,"sourceCode":"        images: Optional[list[\"ImageInput\"]] = None,\n        videos: Optional[list[\"VideoInput\"]] = None,\n        audios: Optional[list[\"AudioInput\"]] = None,\n        **input_kwargs,\n    ) -> AsyncGenerator[str, None]:\n        generated_text = \"\"\n        generator = await self._generate(messages, system, tools, images, videos, audios, **input_kwargs)\n        for result in generator:\n            delta_text = result[\"text\"][len(generated_text) :]\n            generated_text = result[\"text\"]\n            yield delta_text\n\n    @override\n    async def get_scores(\n        self,\n        batch_input: list[str],\n        **input_kwargs,\n    ) -> list[float]:\n        raise NotImplementedError(\"SGLang engine does not support `get_scores`.\")\n\n    def __del__(self):\n        r\"\"\"Ensure server is cleaned up when object is deleted.\"\"\"\n        self._cleanup_server()\n        try:\n            atexit.unregister(self._cleanup_server)\n        except Exception:\n            pass\n","sourceCodeStart":266,"sourceCodeEnd":293,"githubUrl":"https://github.com/hiyouga/LlamaFactory/blob/f28afaf6355af515454dfb16c97d728307c93897/src/llamafactory/chat/sglang_engine.py#L266-L293","documentation":"SGLangEngine.get_scores unconditionally raises NotImplementedError. get_scores is the reward-model / logit-scoring interface used by reward-model evaluation and some preference pipelines; the SGLang backend implements only text generation over HTTP (/generate), not sequence log-prob or reward scoring.","triggerScenarios":"Calling await engine.get_scores(batch_input) on an engine constructed with inference_backend sglang — e.g. running reward model evaluation or PPO reward scoring configured to use the sglang engine.","commonSituations":"Setting the eval/reward backend to sglang in a config that was previously run with the hf engine; scripts that branch on engine type incorrectly and reach get_scores for a chat-only backend.","solutions":["Use the hf engine (or another backend that implements get_scores) for reward scoring / RM evaluation.","Branch on engine capability before calling: hasattr check or engine-type check so sglang never receives get_scores calls."],"exampleFix":"# before\nscores = await engine.get_scores(batch_input)  # engine is SGLangEngine\n\n# after\nif isinstance(engine, SGLangEngine):\n    raise SystemExit(\"use hf engine for scoring\")\nscores = await engine.get_scores(batch_input)","handlingStrategy":"type-guard","validationCode":"from llamafactory.chat.sglang_engine import SGLangEngine\nassert not isinstance(engine, SGLangEngine), \"sglang engine cannot score sequences\"","typeGuard":"def supports_scoring(engine) -> bool:\n    return not type(engine).__name__ == \"SGLangEngine\"  # or check for get_scores impl","tryCatchPattern":null,"preventionTips":["Decide the engine backend from the workload: generation-only vs scoring.","Document per-backend capability matrix in your eval scripts."],"tags":["sglang","not-implemented","reward-model","inference"],"backgroundTag":null,"analyzedSha":"f28afaf6355af515454dfb16c97d728307c93897","analyzedAt":"2026-08-14T21:57:28.298Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}