{"record":{"id":"da6d043eaa9c69a1","repo":"sgl-project/sglang","slug":"return-pooled-hidden-states-is-not-supported-for","errorCode":null,"errorMessage":"return_pooled_hidden_states is not supported for {archs[0]}. This model uses CrossEncodingPooler which does not expose pre-head hidden states.","messagePattern":"return_pooled_hidden_states is not supported for (.+?)\\. This model uses CrossEncodingPooler which does not expose pre-head hidden states\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/managers/tokenizer_manager_score_mixin.py","lineNumber":594,"sourceCode":"                )\n            )\n        else:\n            raise ValueError(\n                \"Invalid combination of query/items types for score_request.\"\n            )\n\n        if return_pooled_hidden_states:\n            if is_generation:\n                raise ValueError(\n                    \"return_pooled_hidden_states is not supported for CausalLM models. \"\n                    \"It requires a model with a task-specific head \"\n                    \"(e.g. SequenceClassification or RewardModel).\"\n                )\n            model_config = self.model_config\n            if model_config is not None:\n                archs = getattr(model_config.hf_config, \"architectures\", []) or []\n                if is_cross_encoding_pooler_model(archs):\n                    raise ValueError(\n                        f\"return_pooled_hidden_states is not supported for \"\n                        f\"{archs[0]}. This model uses CrossEncodingPooler which \"\n                        f\"does not expose pre-head hidden states.\"\n                    )\n\n        # Create the appropriate request type\n        mis_delimiter_indices = [delimiter_indices] if use_multi_item_scoring else None\n        if is_generation:\n            batch_request = GenerateReqInput(\n                text=text_prompts,\n                input_ids=input_ids,\n                token_ids_logprob=label_token_ids,\n                return_logprob=True,\n                # Set logprob_start_len=0 for multi-item scoring since we want logprobs at all delimiter positions\n                logprob_start_len=0 if use_multi_item_scoring else -1,\n                stream=False,\n                sampling_params={\"max_new_tokens\": 0},\n                positional_embed_overrides=positional_embed_overrides,","sourceCodeStart":576,"sourceCodeEnd":612,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/managers/tokenizer_manager_score_mixin.py#L576-L612","documentation":"Some reranking architectures route through CrossEncodingPooler, which only emits final cross-encoder scores and never exposes pre-head pooled hidden states. score_request inspects model_config.hf_config.architectures and rejects return_pooled_hidden_states=True for those architectures.","triggerScenarios":"Calling score(..., return_pooled_hidden_states=True) on a cross-encoder reranker whose architecture is flagged by is_cross_encoding_pooler_model(archs) (e.g. modern cross-encoder rerankers served via the score API).","commonSituations":"Attempting to extract intermediate embeddings from a production reranker (e.g. bge-reranker-style cross encoders); assuming all score-capable models expose pooled states.","solutions":["Use a bi-encoder/embedding model if you need pooled hidden states","Drop the flag and consume the reranker's similarity scores","Switch to a SequenceClassification-style model that does not use CrossEncodingPooler"],"exampleFix":"# before\nout = await engine.async_score(q, d, return_pooled_hidden_states=True)  # cross-encoder\n# after\nout = await engine.async_score(q, d)\nhidden = await embedding_engine.encode(q)","handlingStrategy":"validation","validationCode":"from sglang.srt.configs.model_config import is_cross_encoding_pooler_model  # if exposed\narchs = getattr(engine.model_config.hf_config, \"architectures\", []) or []\nwants_hidden = return_pooled_hidden_states and is_cross_encoding_pooler_model(archs)\n# fall back to scores-only when wants_hidden is True","typeGuard":"def can_return_pooled_hidden(archs: list[str]) -> bool:\n    return not is_cross_encoding_pooler_model(archs)","tryCatchPattern":null,"preventionTips":["Feature-detect architecture support before requesting hidden states","Keep an embedding model endpoint for hidden-state extraction instead of a reranker"],"tags":["sglang","scoring","cross-encoder","hidden-states"],"backgroundTag":"unsupported-model-capability","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}