sgl-project/sglang · error · ValueError

token_ids_logprob must be a flat list of integers.

Error message

token_ids_logprob must be a flat list of integers.

What it means

Raised by TokenizerManager._validate_one_request when a request's token_ids_logprob field is non-empty but not a Python list (e.g. an int, tuple, np.ndarray, or nested list). SGLang requires token_ids_logprob to be a flat list of ints because it is forwarded verbatim to the scheduler for selective logprob capture.

Source

Thrown at python/sglang/srt/managers/tokenizer_manager.py:1308

                f"Model '{self.model_config.model_path}' only supports {self.model_config.matryoshka_dimensions} matryoshka dimensions, "
                f"using other output dimensions will lead to poor results."
            )

        if obj.dimensions > self.model_config.hidden_size:
            raise ValueError(
                f"Provided dimensions are greater than max embedding dimension: {self.model_config.hidden_size}"
            )

    def _validate_token_ids_logprob(self, obj: GenerateReqInput) -> None:
        # Batch requests are split into per-request sub-objects before this
        # runs (normalize_batch_and_arguments + __getitem__), so the only
        # legal shape here is the per-request contract of
        # TokenizedGenerateReqInput.token_ids_logprob: a flat list of ints.
        token_ids_logprob = obj.token_ids_logprob
        if not token_ids_logprob:
            return
        if not isinstance(token_ids_logprob, list):
            raise ValueError("token_ids_logprob must be a flat list of integers.")
        vocab_size = self.model_config.vocab_size
        for token_id in token_ids_logprob:
            if not isinstance(token_id, int):
                raise ValueError("token_ids_logprob must be a flat list of integers.")
            if token_id < 0 or token_id >= vocab_size:
                raise ValueError(
                    f"token_ids_logprob contains out-of-vocabulary token id "
                    f"{token_id}; valid range is [0, {vocab_size})."
                )

    def _validate_input_ids_in_vocab(
        self, input_ids: Union[List[int], List[List[int]]], vocab_size: int
    ) -> None:
        # Handle both single sequence and batch of sequences
        if isinstance(input_ids[0], list):
            # Batch of sequences
            for seq in input_ids:
                if any(id >= vocab_size for id in seq):

View on GitHub (pinned to 0132848349)

Solutions

1. Convert to a flat Python list: token_ids_logprob=list(np.asarray(ids).ravel()) or ids.tolist()
2. For batch requests, supply one flat list per request object rather than a nested list
3. Ensure every element is a Python int (not np.integer or str) before sending

Example fix

// before
req = GenerateReqInput(text="hi", token_ids_logprob=np.array([1,2,3]))
// after
req = GenerateReqInput(text="hi", token_ids_logprob=[1, 2, 3])

if token_ids_logprob is not None and (not isinstance(token_ids_logprob, list) or not all(isinstance(t, int) for t in token_ids_logprob)):
    token_ids_logprob = [int(t) for t in np.ravel(token_ids_logprob).tolist()]

def is_flat_int_list(v) -> bool:
    return isinstance(v, list) and all(isinstance(t, int) and not isinstance(t, bool) for t in v)

except ValueError as e: assert 'flat list of integers' in str(e); fix payload client-side and retry once

When it happens

Trigger: Calling /generate or LLM.generate with token_ids_logprob set to a tuple, numpy array, or a list-of-lists (batch form) instead of a flat Python list of token ids.

Common situations: Building requests from numpy token arrays without .tolist(), using the batch [[...],[...]] shape for a single request, or serializing through a client that converts lists to tuples.

Related errors

• v_cache must be provided
• q can only be None when only_qv=True
• q must be provided unless qv is provided with only_qv=True
• name must be provided
• Encoded prompt has {tensor.shape[1]} tokens, expected at lea

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/8a70b27bc5423285. Report an issue: GitHub.