{"record":{"id":"4f4e12417733200a","repo":"sgl-project/sglang","slug":"return-flat-raw-top-logprobs-requires-rectangular","errorCode":null,"errorMessage":"return_flat_raw_top_logprobs requires rectangular top logprob rows with nulls only in the leading prefix; row {null_prefix + offset} has {None if row is None else len(row)} entries (expected {k}).","messagePattern":"return_flat_raw_top_logprobs requires rectangular top logprob rows with nulls only in the leading prefix; row (.+?) has (.+?) entries \\(expected (.+?)\\)\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/managers/io_struct.py","lineNumber":1409,"sourceCode":"    \"\"\"Convert nested per-position prompt top logprob rows into the flat\n    arrays of the `return_flat_raw_top_logprobs` response format.\n\n    Returns (float32 values [rows, k], int32 token ids [rows, k],\n    null_prefix). The leading null rows are counted into null_prefix and\n    excluded from the arrays. Raises ValueError when the rows are not\n    representable by (shape, null_prefix): interior nulls or ragged k,\n    e.g. multi-item scoring.\n    \"\"\"\n    num_rows = len(input_top_logprobs_val)\n    null_prefix = 0\n    while null_prefix < num_rows and not input_top_logprobs_val[null_prefix]:\n        null_prefix += 1\n    val_rows = input_top_logprobs_val[null_prefix:]\n    idx_rows = input_top_logprobs_idx[null_prefix:]\n    k = len(val_rows[0]) if val_rows else top_logprobs_num\n    for offset, row in enumerate(val_rows):\n        if row is None or len(row) != k:\n            raise ValueError(\n                \"return_flat_raw_top_logprobs requires rectangular top logprob \"\n                f\"rows with nulls only in the leading prefix; row {null_prefix + offset} \"\n                f\"has {None if row is None else len(row)} entries (expected {k}).\"\n            )\n    val_arr = np.asarray(val_rows, dtype=np.float32).reshape(len(val_rows), k)\n    idx_arr = np.asarray(idx_rows, dtype=np.int32).reshape(len(idx_rows), k)\n    return val_arr, idx_arr, null_prefix\n\n\nclass BatchTokenIDOutput(BaseBatchReq, kw_only=True):\n    # The finish reason\n    finished_reasons: List[Optional[FinishReasonDict]]\n    # For incremental decoding\n    decoded_texts: List[str]\n    decode_ids: List[array]  # List[array[int]]\n    read_offsets: List[int]\n    # Only used when `--skip-tokenizer-init` is on\n    output_ids: Optional[List[array]]  # Optional[List[array[int]]]","sourceCodeStart":1391,"sourceCodeEnd":1427,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/managers/io_struct.py#L1391-L1427","documentation":"With return_flat_raw_top_logprobs, input top-logprob rows must form a rectangular matrix: rows may be None only as a leading prefix (prompt positions before sampling), after which every row must have exactly k entries (k = first non-null row's length). Any None or ragged row after the prefix breaks the numpy reshape.","triggerScenarios":"Building input_top_logprobs_val with rows of differing lengths, e.g. [[a,b],[a,b,c]], or a None row sandwiched between valid rows.","commonSituations":"Client code assembling logprob rows per prompt token with a variable top-k per position; upstream data produced by a different top_logprobs_num than requested.","solutions":["Use a constant k = top_logprobs_num for every row","Pad short rows (e.g. with None-fillers converted per protocol) or trim to k","Move all None rows to the leading prefix only"],"exampleFix":"// before\ninput_top_logprobs_val=[[0.1,0.2],[0.3]]\n// after\ninput_top_logprobs_val=[[0.1,0.2],[0.3,0.05]]  # pad to k=2","handlingStrategy":"validation","validationCode":"rows = [r for r in input_top_logprobs_val if r is not None]\nassert rows and all(len(r) == top_logprobs_num for r in rows)\nassert all(r is None for r in rows_before_first_valid)","typeGuard":"def is_rectangular_with_leading_nulls(rows, k):\n    seen = False\n    for r in rows:\n        if r is None:\n            if seen: return False\n        else:\n            seen = True\n            if len(r) != k: return False\n    return True","tryCatchPattern":null,"preventionTips":["Always emit exactly top_logprobs_num entries per row","Treat None rows as prefix-only semantics"],"tags":["sglang","logprobs","data-shape","input-validation"],"backgroundTag":"ragged-array-shape-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}