sgl-project/sglang · error · ValueError

return_hidden_states_mode must be one of: None, 'last', or '

Error message

return_hidden_states_mode must be one of: None, 'last', or 'full'.

What it means

The return_hidden_states_mode field only accepts None, 'last', or 'full'. The resolution handler _handle_return_hidden_states_mode validates it early because downstream code branches on these exact values to collect hidden states for the request response.

Source

Thrown at python/sglang/srt/server_args.py:4007

        self._handle_debug_utils()

        # Handle any other necessary validations.
        self._handle_other_validations()

        # Model-capability adjustments that legacy code applied at model-load
        # time; last declarations of the resolution, mirroring that order.
        self._handle_model_capability_adjustments()

        # Validate after all batch-size declarations are visible.
        self._validate_deepep_v2_speculative_draft()
        self._validate_deepep_v2_dispatch_token_budget()

        self._resolution_finished = True

    def _handle_return_hidden_states_mode(self):
        cfg = resolving_view(self)
        if cfg.return_hidden_states_mode not in (None, "last", "full"):
            raise ValueError(
                "return_hidden_states_mode must be one of: None, 'last', or 'full'."
            )
        if cfg.return_hidden_states_mode is None:
            if cfg.enable_return_hidden_states:
                self._declare(
                    "_handle_return_hidden_states_mode",
                    return_hidden_states_mode="full",
                )
        else:
            self._declare(
                "_handle_return_hidden_states_mode",
                enable_return_hidden_states=True,
            )

    def _handle_model_capability_adjustments(self):
        cfg = resolving_view(self)
        if parse_connector_type(cfg.model_path) == ConnectorType.INSTANCE:
            return

View on GitHub (pinned to 0132848349)

Solutions

1. Use one of: omit it (None), 'last', or 'full'.
2. If you only need the final layer's last-token hidden state, use 'last'; for all layers' hidden states use 'full'.
3. Check for typos/case in CLI or YAML configs.

Example fix

# before
python -m sglang.launch_server --model m --return-hidden-states-mode all
# after
python -m sglang.launch_server --model m --return-hidden-states-mode full

assert return_hidden_states_mode in (None, 'last', 'full'), return_hidden_states_mode

def valid_hidden_states_mode(v) -> bool:
    return v in (None, 'last', 'full')

When it happens

Trigger: Passing --return-hidden-states-mode with a typo'd or unsupported value (e.g. "all", "LAST", "last_token", "") in the CLI, ServerArgs, or an env-derived config.

Common situations: Assuming 'all' is the plural form; case mismatch; passing an empty string instead of omitting the flag; copy-pasting from vLLM-style option names.

Understand the failure class

Background: Invalid enum value errors: "Unknown type", "Invalid scope", "must be one of" — when a string is not on the library's allowed list — this error's family across 23 libraries.

Related errors

• Unsupported text encoder output: expected `hidden_states`.
• return_hidden_states must be a boolean or the string literal
• return_pooled_hidden_states is not supported for CausalLM mo
• return_pooled_hidden_states is not supported for {archs[0]}.
• resolution already failed on this ServerArgs; the handlers t

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/c082b5dd832b9e72. Report an issue: GitHub.