sgl-project/sglang · error · ValueError

label_token_ids is required for generation (CausalLM) models

Error message

label_token_ids is required for generation (CausalLM) models.

What it means

score_request for generation (CausalLM) models requires label_token_ids, since scoring with a generative model works by computing log-prob/scores against provided label tokens. The method reads self.is_generation and rejects the call when labels are missing.

Source

Thrown at python/sglang/srt/managers/tokenizer_manager_score_mixin.py:480

           multiple items into a single sequence using delimiter for efficient processing.
           Note: item_first parameter is ignored in multi-item scoring mode since it uses
           a fixed format: query<delimiter>item1<delimiter>item2<delimiter>item3<delimiter>

           Multi-item scoring works with both text and pre-tokenized inputs:
           - Text: query<delimiter_text>item1<delimiter_text>item2<delimiter_text>item3<delimiter_text>
           - Tokens: query<delimiter_token_id>item1<delimiter_token_id>item2<delimiter_token_id>item3<delimiter_token_id>

        Supports two model types:
        - Generation (CausalLM): Requires label_token_ids; returns logprob-based scores.
        - SequenceClassification: label_token_ids is optional; returns pooled class logits.

        return_pooled_hidden_states is only supported for non-generation models
        (SequenceClassification, RewardModel); raises ValueError for CausalLM.
        """
        is_generation = self.is_generation

        if is_generation and label_token_ids is None:
            raise ValueError(
                "label_token_ids is required for generation (CausalLM) models."
            )
        if items is None:
            raise ValueError("items must be provided")
        if not items:
            return ScoreResult(scores=[], prompt_tokens=0)

        has_embeds = (
            query_embed_overrides is not None or item_embed_overrides is not None
        )
        if has_embeds and embed_override_token_id is None:
            raise ValueError(
                "embed_override_token_id is required when query_embed_overrides "
                "or item_embed_overrides are supplied."
            )
        if item_first and has_embeds:
            raise ValueError("item_first is not supported when embeddings are supplied")
        if item_embed_overrides is not None and len(item_embed_overrides) != len(items):

View on GitHub (pinned to 0132848349)

Solutions

Supply label_token_ids (and label_token_ids as str accepts a list of ids) matching what you want scored
Or switch to a scoring-oriented architecture (SequenceClassification / RewardModel) if you want label-free scoring

Example fix

# before
result = engine.score_request(query="Q", items=["A"], ...)

# after
result = engine.score_request(query="Q", items=["A"],
    label_token_ids=tokenizer.encode("A") ...)

Defensive patterns

Strategy: validation

Validate before calling

if engine.is_generation and label_token_ids is None:
    raise ValueError("provide label_token_ids for CausalLM scoring")

Type guard

def can_score_without_labels(engine) -> bool:
    return not getattr(engine, "is_generation", True)

Try / catch

try:
    r = engine.score_request(query=q, items=items)
except ValueError as e:
    if "label_token_ids is required" in str(e):
        r = engine.score_request(query=q, items=items, label_token_ids=labels)
    else:
        raise

Prevention

Check /get_server_info model type before writing scoring clients
For CausalLM scoring always construct label_token_ids from the target text
Use SequenceClassification/Reward checkpoints for label-free scoring

When it happens

Trigger: Calling engine.score / score_request without label_token_ids while the loaded model path/architecture is a CausalLM (is_generation True).

Common situations: Porting scoring code written for a SequenceClassification/RewardModel server to a CausalLM endpoint; assuming classifier-style zero-label scoring works on generative models; config flag mixups identifying the model as generative.

Understand the failure class

Background: Missing required parameter errors: what 'X is required' and 'the required X param is missing' mean, and how to fix them — this error's family across 27 libraries.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/febe718cf3637e16. Report an issue: GitHub.