sgl-project/sglang · error · ValueError

return_hidden_states must be a boolean or the string literal

Error message

return_hidden_states must be a boolean or the string literal 'last'.

What it means

get_return_hidden_states_mode maps return_hidden_states to a CaptureHiddenMode: True->FULL, 'last'->LAST, False->NULL. Any other value (e.g. 'all', 1, 'full') is rejected because hidden-state capture only supports these exact modes.

Source

Thrown at python/sglang/srt/managers/schedule_batch.py:177

MM_PAD_SHIFT_VALUE = 1_000_000
_MM_HASH_MASK = (1 << 64) - 1

logger = logging.getLogger(__name__)


ReturnHiddenStatesMode = Union[bool, Literal["last"]]


def get_return_hidden_states_mode(
    return_hidden_states: ReturnHiddenStatesMode,
) -> CaptureHiddenMode:
    if return_hidden_states is True:
        return CaptureHiddenMode.FULL
    if return_hidden_states == "last":
        return CaptureHiddenMode.LAST
    if return_hidden_states is False:
        return CaptureHiddenMode.NULL
    raise ValueError(
        "return_hidden_states must be a boolean or the string literal 'last'."
    )


def get_request_return_hidden_states_mode(
    return_hidden_states: Union[List[ReturnHiddenStatesMode], ReturnHiddenStatesMode],
) -> CaptureHiddenMode:
    if isinstance(return_hidden_states, list):
        return max(
            (get_return_hidden_states_mode(mode) for mode in return_hidden_states),
            default=CaptureHiddenMode.NULL,
        )
    return get_return_hidden_states_mode(return_hidden_states)


def get_batch_return_hidden_states_mode(reqs: List[Req]) -> CaptureHiddenMode:
    mode = CaptureHiddenMode.NULL
    for req in reqs:

View on GitHub (pinned to 0132848349)

Solutions

1. Use True for all layers' hidden states, 'last' for final layer, False for none

Example fix

// before
return_hidden_states='all'
// after
return_hidden_states=True

assert return_hidden_states in (True, False, 'last')

def valid_rhs(v): return v is True or v is False or v == 'last'

When it happens

Trigger: GenerateReqInput(..., return_hidden_states='all') or =1; REST body {"return_hidden_states": "full"}.

Common situations: Users assuming an OpenAI-style string like 'all'; passing truthy ints from JSON clients.

Understand the failure class

Background: Invalid enum value errors: "Unknown type", "Invalid scope", "must be one of" — when a string is not on the library's allowed list — this error's family across 23 libraries.

Related errors

• Unsupported text encoder output: expected `hidden_states`.
• MiniMax H3 model variant must be a non-empty string
• Unknown input type: {response_msg['type']}
• Every extra_key should be a string.
• extra_key should be a list or a string.

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/f95da9abf11c5e44. Report an issue: GitHub.