sgl-project/sglang · error · ValueError

return_flat_raw_top_logprobs requires rectangular top logpro

Error message

return_flat_raw_top_logprobs requires rectangular top logprob rows with nulls only in the leading prefix; row {null_prefix + offset} has {None if row is None else len(row)} entries (expected {k}).

What it means

With return_flat_raw_top_logprobs, input top-logprob rows must form a rectangular matrix: rows may be None only as a leading prefix (prompt positions before sampling), after which every row must have exactly k entries (k = first non-null row's length). Any None or ragged row after the prefix breaks the numpy reshape.

Source

Thrown at python/sglang/srt/managers/io_struct.py:1409

    """Convert nested per-position prompt top logprob rows into the flat
    arrays of the `return_flat_raw_top_logprobs` response format.

    Returns (float32 values [rows, k], int32 token ids [rows, k],
    null_prefix). The leading null rows are counted into null_prefix and
    excluded from the arrays. Raises ValueError when the rows are not
    representable by (shape, null_prefix): interior nulls or ragged k,
    e.g. multi-item scoring.
    """
    num_rows = len(input_top_logprobs_val)
    null_prefix = 0
    while null_prefix < num_rows and not input_top_logprobs_val[null_prefix]:
        null_prefix += 1
    val_rows = input_top_logprobs_val[null_prefix:]
    idx_rows = input_top_logprobs_idx[null_prefix:]
    k = len(val_rows[0]) if val_rows else top_logprobs_num
    for offset, row in enumerate(val_rows):
        if row is None or len(row) != k:
            raise ValueError(
                "return_flat_raw_top_logprobs requires rectangular top logprob "
                f"rows with nulls only in the leading prefix; row {null_prefix + offset} "
                f"has {None if row is None else len(row)} entries (expected {k})."
            )
    val_arr = np.asarray(val_rows, dtype=np.float32).reshape(len(val_rows), k)
    idx_arr = np.asarray(idx_rows, dtype=np.int32).reshape(len(idx_rows), k)
    return val_arr, idx_arr, null_prefix


class BatchTokenIDOutput(BaseBatchReq, kw_only=True):
    # The finish reason
    finished_reasons: List[Optional[FinishReasonDict]]
    # For incremental decoding
    decoded_texts: List[str]
    decode_ids: List[array]  # List[array[int]]
    read_offsets: List[int]
    # Only used when `--skip-tokenizer-init` is on
    output_ids: Optional[List[array]]  # Optional[List[array[int]]]

View on GitHub (pinned to 0132848349)

Solutions

1. Use a constant k = top_logprobs_num for every row
2. Pad short rows (e.g. with None-fillers converted per protocol) or trim to k
3. Move all None rows to the leading prefix only

Example fix

// before
input_top_logprobs_val=[[0.1,0.2],[0.3]]
// after
input_top_logprobs_val=[[0.1,0.2],[0.3,0.05]]  # pad to k=2

rows = [r for r in input_top_logprobs_val if r is not None]
assert rows and all(len(r) == top_logprobs_num for r in rows)
assert all(r is None for r in rows_before_first_valid)

def is_rectangular_with_leading_nulls(rows, k):
    seen = False
    for r in rows:
        if r is None:
            if seen: return False
        else:
            seen = True
            if len(r) != k: return False
    return True

When it happens

Trigger: Building input_top_logprobs_val with rows of differing lengths, e.g. [[a,b],[a,b,c]], or a None row sandwiched between valid rows.

Common situations: Client code assembling logprob rows per prompt token with a variable top-k per position; upstream data produced by a different top_logprobs_num than requested.

Related errors

• MiniMax H3 model variant must be a non-empty string
• top_logprobs_num {top_logprobs_len} exceeds disaggregation m
• Unknown input type: {response_msg['type']}
• Every extra_key should be a string.
• extra_key should be a list or a string.

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/4f4e12417733200a. Report an issue: GitHub.