{"record":{"id":"e7de7500a6c35a82","repo":"sgl-project/sglang","slug":"return-pooled-hidden-states-is-not-supported-for-c","errorCode":null,"errorMessage":"return_pooled_hidden_states is not supported for CausalLM models. It requires a model with a task-specific head (e.g. SequenceClassification or RewardModel).","messagePattern":"return_pooled_hidden_states is not supported for CausalLM models\\. It requires a model with a task-specific head \\(e\\.g\\. SequenceClassification or RewardModel\\)\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/managers/tokenizer_manager_score_mixin.py","lineNumber":585,"sourceCode":"            _, input_ids, positional_embed_overrides, delimiter_indices = (\n                self._build_token_id_inputs(\n                    query_ids,\n                    items_ids,\n                    item_first,\n                    use_multi_item_scoring,\n                    embed_override_token_id,\n                    query_embed_overrides,\n                    item_embed_overrides,\n                )\n            )\n        else:\n            raise ValueError(\n                \"Invalid combination of query/items types for score_request.\"\n            )\n\n        if return_pooled_hidden_states:\n            if is_generation:\n                raise ValueError(\n                    \"return_pooled_hidden_states is not supported for CausalLM models. \"\n                    \"It requires a model with a task-specific head \"\n                    \"(e.g. SequenceClassification or RewardModel).\"\n                )\n            model_config = self.model_config\n            if model_config is not None:\n                archs = getattr(model_config.hf_config, \"architectures\", []) or []\n                if is_cross_encoding_pooler_model(archs):\n                    raise ValueError(\n                        f\"return_pooled_hidden_states is not supported for \"\n                        f\"{archs[0]}. This model uses CrossEncodingPooler which \"\n                        f\"does not expose pre-head hidden states.\"\n                    )\n\n        # Create the appropriate request type\n        mis_delimiter_indices = [delimiter_indices] if use_multi_item_scoring else None\n        if is_generation:\n            batch_request = GenerateReqInput(","sourceCodeStart":567,"sourceCodeEnd":603,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/managers/tokenizer_manager_score_mixin.py#L567-L603","documentation":"return_pooled_hidden_states asks the scoring path to return pre-head pooled hidden states, but a CausalLM (generation) model has no task-specific head, so the flag only applies to models like SequenceClassification or RewardModel served through the score API.","triggerScenarios":"Launching the engine on a plain causal LM (is_generation=True) and calling score(..., return_pooled_hidden_states=True).","commonSituations":"Trying to extract embeddings from a base chat model via the score endpoint instead of an embedding/reranking model; reusing a script written for a RewardModel against a CausalLM checkpoint; missing --task-type override when loading a sequence-classification checkpoint.","solutions":["Switch to a model with a scoring head (SequenceClassification/RewardModel architectures)","Drop return_pooled_hidden_states=True if you only need scores","If the checkpoint is a seq-cls model, ensure server args/task type set it as non-generation so is_generation is False"],"exampleFix":"# before\nres = await engine.async_score(q, d, return_pooled_hidden_states=True)  # CausalLM server\n# after\nres = await engine.async_score(q, d)","handlingStrategy":"validation","validationCode":"archs = getattr(engine.model_config.hf_config, \"architectures\", [])\nif return_pooled_hidden_states and engine.is_generation:\n    return_pooled_hidden_states = False  # or raise early with clear context","typeGuard":"def supports_pooled_hidden_states(engine) -> bool:\n    return not getattr(engine, \"is_generation\", True)","tryCatchPattern":null,"preventionTips":["Only expose return_pooled_hidden_states in wrappers targeting SequenceClassification/Reward models","Check the served model's task type before enabling the flag"],"tags":["sglang","scoring","hidden-states","model-architecture"],"backgroundTag":"unsupported-model-capability","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}