hiyouga/LlamaFactory · error · NotImplementedError

SGLang engine does not support `get_scores`.

Error message

SGLang engine does not support `get_scores`.

What it means

SGLangEngine.get_scores unconditionally raises NotImplementedError. get_scores is the reward-model / logit-scoring interface used by reward-model evaluation and some preference pipelines; the SGLang backend implements only text generation over HTTP (/generate), not sequence log-prob or reward scoring.

Source

Thrown at src/llamafactory/chat/sglang_engine.py:284

        images: Optional[list["ImageInput"]] = None,
        videos: Optional[list["VideoInput"]] = None,
        audios: Optional[list["AudioInput"]] = None,
        **input_kwargs,
    ) -> AsyncGenerator[str, None]:
        generated_text = ""
        generator = await self._generate(messages, system, tools, images, videos, audios, **input_kwargs)
        for result in generator:
            delta_text = result["text"][len(generated_text) :]
            generated_text = result["text"]
            yield delta_text

    @override
    async def get_scores(
        self,
        batch_input: list[str],
        **input_kwargs,
    ) -> list[float]:
        raise NotImplementedError("SGLang engine does not support `get_scores`.")

    def __del__(self):
        r"""Ensure server is cleaned up when object is deleted."""
        self._cleanup_server()
        try:
            atexit.unregister(self._cleanup_server)
        except Exception:
            pass

View on GitHub (pinned to f28afaf635)

Solutions

1. Use the hf engine (or another backend that implements get_scores) for reward scoring / RM evaluation.
2. Branch on engine capability before calling: hasattr check or engine-type check so sglang never receives get_scores calls.

Example fix

# before
scores = await engine.get_scores(batch_input)  # engine is SGLangEngine

# after
if isinstance(engine, SGLangEngine):
    raise SystemExit("use hf engine for scoring")
scores = await engine.get_scores(batch_input)

from llamafactory.chat.sglang_engine import SGLangEngine
assert not isinstance(engine, SGLangEngine), "sglang engine cannot score sequences"

def supports_scoring(engine) -> bool:
    return not type(engine).__name__ == "SGLangEngine"  # or check for get_scores impl

When it happens

Trigger: Calling await engine.get_scores(batch_input) on an engine constructed with inference_backend sglang — e.g. running reward model evaluation or PPO reward scoring configured to use the sglang engine.

Common situations: Setting the eval/reward backend to sglang in a config that was previously run with the hf engine; scripts that branch on engine type incorrectly and reach get_scores for a chat-only backend.

Related errors

• SGLang only supports n=1.
• vLLM engine does not support `get_scores`.
• SGLang not install, you may need to run `pip install sglang[
• SGLang server initialization failed: {str(e)}.
• SGLang server error: {response.status_code}, {response.text}

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/6ebd52aee1985f90. Report an issue: GitHub.