sgl-project/sglang · error · ValueError

return_pooled_hidden_states is not supported for {archs[0]}.

Error message

return_pooled_hidden_states is not supported for {archs[0]}. This model uses CrossEncodingPooler which does not expose pre-head hidden states.

What it means

Some reranking architectures route through CrossEncodingPooler, which only emits final cross-encoder scores and never exposes pre-head pooled hidden states. score_request inspects model_config.hf_config.architectures and rejects return_pooled_hidden_states=True for those architectures.

Source

Thrown at python/sglang/srt/managers/tokenizer_manager_score_mixin.py:594

                )
            )
        else:
            raise ValueError(
                "Invalid combination of query/items types for score_request."
            )

        if return_pooled_hidden_states:
            if is_generation:
                raise ValueError(
                    "return_pooled_hidden_states is not supported for CausalLM models. "
                    "It requires a model with a task-specific head "
                    "(e.g. SequenceClassification or RewardModel)."
                )
            model_config = self.model_config
            if model_config is not None:
                archs = getattr(model_config.hf_config, "architectures", []) or []
                if is_cross_encoding_pooler_model(archs):
                    raise ValueError(
                        f"return_pooled_hidden_states is not supported for "
                        f"{archs[0]}. This model uses CrossEncodingPooler which "
                        f"does not expose pre-head hidden states."
                    )

        # Create the appropriate request type
        mis_delimiter_indices = [delimiter_indices] if use_multi_item_scoring else None
        if is_generation:
            batch_request = GenerateReqInput(
                text=text_prompts,
                input_ids=input_ids,
                token_ids_logprob=label_token_ids,
                return_logprob=True,
                # Set logprob_start_len=0 for multi-item scoring since we want logprobs at all delimiter positions
                logprob_start_len=0 if use_multi_item_scoring else -1,
                stream=False,
                sampling_params={"max_new_tokens": 0},
                positional_embed_overrides=positional_embed_overrides,

View on GitHub (pinned to 0132848349)

Solutions

1. Use a bi-encoder/embedding model if you need pooled hidden states
2. Drop the flag and consume the reranker's similarity scores
3. Switch to a SequenceClassification-style model that does not use CrossEncodingPooler

Example fix

# before
out = await engine.async_score(q, d, return_pooled_hidden_states=True)  # cross-encoder
# after
out = await engine.async_score(q, d)
hidden = await embedding_engine.encode(q)

from sglang.srt.configs.model_config import is_cross_encoding_pooler_model  # if exposed
archs = getattr(engine.model_config.hf_config, "architectures", []) or []
wants_hidden = return_pooled_hidden_states and is_cross_encoding_pooler_model(archs)
# fall back to scores-only when wants_hidden is True

def can_return_pooled_hidden(archs: list[str]) -> bool:
    return not is_cross_encoding_pooler_model(archs)

When it happens

Trigger: Calling score(..., return_pooled_hidden_states=True) on a cross-encoder reranker whose architecture is flagged by is_cross_encoding_pooler_model(archs) (e.g. modern cross-encoder rerankers served via the score API).

Common situations: Attempting to extract intermediate embeddings from a production reranker (e.g. bge-reranker-style cross encoders); assuming all score-capable models expose pooled states.

Related errors

• return_pooled_hidden_states is not supported for CausalLM mo
• Unsupported text encoder output: expected `hidden_states`.
• return_hidden_states must be a boolean or the string literal
• item_first is not supported when embeddings are supplied
• item_embed_overrides length ({len(item_embed_overrides)}) mu

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/da6d043eaa9c69a1. Report an issue: GitHub.