{"record":{"id":"8a70b27bc5423285","repo":"sgl-project/sglang","slug":"token-ids-logprob-must-be-a-flat-list-of-integers","errorCode":null,"errorMessage":"token_ids_logprob must be a flat list of integers.","messagePattern":"token_ids_logprob must be a flat list of integers\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/managers/tokenizer_manager.py","lineNumber":1308,"sourceCode":"                f\"Model '{self.model_config.model_path}' only supports {self.model_config.matryoshka_dimensions} matryoshka dimensions, \"\n                f\"using other output dimensions will lead to poor results.\"\n            )\n\n        if obj.dimensions > self.model_config.hidden_size:\n            raise ValueError(\n                f\"Provided dimensions are greater than max embedding dimension: {self.model_config.hidden_size}\"\n            )\n\n    def _validate_token_ids_logprob(self, obj: GenerateReqInput) -> None:\n        # Batch requests are split into per-request sub-objects before this\n        # runs (normalize_batch_and_arguments + __getitem__), so the only\n        # legal shape here is the per-request contract of\n        # TokenizedGenerateReqInput.token_ids_logprob: a flat list of ints.\n        token_ids_logprob = obj.token_ids_logprob\n        if not token_ids_logprob:\n            return\n        if not isinstance(token_ids_logprob, list):\n            raise ValueError(\"token_ids_logprob must be a flat list of integers.\")\n        vocab_size = self.model_config.vocab_size\n        for token_id in token_ids_logprob:\n            if not isinstance(token_id, int):\n                raise ValueError(\"token_ids_logprob must be a flat list of integers.\")\n            if token_id < 0 or token_id >= vocab_size:\n                raise ValueError(\n                    f\"token_ids_logprob contains out-of-vocabulary token id \"\n                    f\"{token_id}; valid range is [0, {vocab_size}).\"\n                )\n\n    def _validate_input_ids_in_vocab(\n        self, input_ids: Union[List[int], List[List[int]]], vocab_size: int\n    ) -> None:\n        # Handle both single sequence and batch of sequences\n        if isinstance(input_ids[0], list):\n            # Batch of sequences\n            for seq in input_ids:\n                if any(id >= vocab_size for id in seq):","sourceCodeStart":1290,"sourceCodeEnd":1326,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/managers/tokenizer_manager.py#L1290-L1326","documentation":"Raised by TokenizerManager._validate_one_request when a request's token_ids_logprob field is non-empty but not a Python list (e.g. an int, tuple, np.ndarray, or nested list). SGLang requires token_ids_logprob to be a flat list of ints because it is forwarded verbatim to the scheduler for selective logprob capture.","triggerScenarios":"Calling /generate or LLM.generate with token_ids_logprob set to a tuple, numpy array, or a list-of-lists (batch form) instead of a flat Python list of token ids.","commonSituations":"Building requests from numpy token arrays without .tolist(), using the batch [[...],[...]] shape for a single request, or serializing through a client that converts lists to tuples.","solutions":["Convert to a flat Python list: token_ids_logprob=list(np.asarray(ids).ravel()) or ids.tolist()","For batch requests, supply one flat list per request object rather than a nested list","Ensure every element is a Python int (not np.integer or str) before sending"],"exampleFix":"// before\nreq = GenerateReqInput(text=\"hi\", token_ids_logprob=np.array([1,2,3]))\n// after\nreq = GenerateReqInput(text=\"hi\", token_ids_logprob=[1, 2, 3])","handlingStrategy":"validation","validationCode":"if token_ids_logprob is not None and (not isinstance(token_ids_logprob, list) or not all(isinstance(t, int) for t in token_ids_logprob)):\n    token_ids_logprob = [int(t) for t in np.ravel(token_ids_logprob).tolist()]","typeGuard":"def is_flat_int_list(v) -> bool:\n    return isinstance(v, list) and all(isinstance(t, int) and not isinstance(t, bool) for t in v)","tryCatchPattern":"except ValueError as e: assert 'flat list of integers' in str(e); fix payload client-side and retry once","preventionTips":["Always .tolist() numpy arrays before building requests","Use one flat list per request; never nest for batch"],"tags":["sglang","validation","logprob","request-validation"],"backgroundTag":"request-schema-validation-failed","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}