sgl-project/sglang · error · ValueError

return_pooled_hidden_states is not supported for CausalLM mo

Error message

return_pooled_hidden_states is not supported for CausalLM models. It requires a model with a task-specific head (e.g. SequenceClassification or RewardModel).

What it means

return_pooled_hidden_states asks the scoring path to return pre-head pooled hidden states, but a CausalLM (generation) model has no task-specific head, so the flag only applies to models like SequenceClassification or RewardModel served through the score API.

Source

Thrown at python/sglang/srt/managers/tokenizer_manager_score_mixin.py:585

            _, input_ids, positional_embed_overrides, delimiter_indices = (
                self._build_token_id_inputs(
                    query_ids,
                    items_ids,
                    item_first,
                    use_multi_item_scoring,
                    embed_override_token_id,
                    query_embed_overrides,
                    item_embed_overrides,
                )
            )
        else:
            raise ValueError(
                "Invalid combination of query/items types for score_request."
            )

        if return_pooled_hidden_states:
            if is_generation:
                raise ValueError(
                    "return_pooled_hidden_states is not supported for CausalLM models. "
                    "It requires a model with a task-specific head "
                    "(e.g. SequenceClassification or RewardModel)."
                )
            model_config = self.model_config
            if model_config is not None:
                archs = getattr(model_config.hf_config, "architectures", []) or []
                if is_cross_encoding_pooler_model(archs):
                    raise ValueError(
                        f"return_pooled_hidden_states is not supported for "
                        f"{archs[0]}. This model uses CrossEncodingPooler which "
                        f"does not expose pre-head hidden states."
                    )

        # Create the appropriate request type
        mis_delimiter_indices = [delimiter_indices] if use_multi_item_scoring else None
        if is_generation:
            batch_request = GenerateReqInput(

View on GitHub (pinned to 0132848349)

Solutions

1. Switch to a model with a scoring head (SequenceClassification/RewardModel architectures)
2. Drop return_pooled_hidden_states=True if you only need scores
3. If the checkpoint is a seq-cls model, ensure server args/task type set it as non-generation so is_generation is False

Example fix

# before
res = await engine.async_score(q, d, return_pooled_hidden_states=True)  # CausalLM server
# after
res = await engine.async_score(q, d)

archs = getattr(engine.model_config.hf_config, "architectures", [])
if return_pooled_hidden_states and engine.is_generation:
    return_pooled_hidden_states = False  # or raise early with clear context

def supports_pooled_hidden_states(engine) -> bool:
    return not getattr(engine, "is_generation", True)

When it happens

Trigger: Launching the engine on a plain causal LM (is_generation=True) and calling score(..., return_pooled_hidden_states=True).

Common situations: Trying to extract embeddings from a base chat model via the score endpoint instead of an embedding/reranking model; reusing a script written for a RewardModel against a CausalLM checkpoint; missing --task-type override when loading a sequence-classification checkpoint.

Related errors

• return_pooled_hidden_states is not supported for {archs[0]}.
• Unsupported text encoder output: expected `hidden_states`.
• return_hidden_states must be a boolean or the string literal
• item_first is not supported when embeddings are supplied
• item_embed_overrides length ({len(item_embed_overrides)}) mu

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/e7de7500a6c35a82. Report an issue: GitHub.