databendlabs/databend · error

transfer target state offsets overflow

Error message

transfer target state offsets overflow

What it means

In commit(), the destination page's state_offsets is advanced via checked_add(offset). This panic fires when that addition overflows the offset integer type, meaning the accumulated state offsets in the target page exceed what the type can represent.

Solutions

  1. Check the offset accounting for double-counting of transferred state sizes.
  2. Reduce per-query aggregate state cardinality (fewer groups / pre-aggregation) to keep offsets within range.
  3. Widen the state_offsets representation in the payload page if legitimately huge transfers are expected.
  4. Report the query; overflow at this point usually signals bookkeeping corruption.
Defensive patterns

Strategy: validation

Validate before calling

if page.state_offsets.checked_add(offset).is_none() {
    return Err(ErrorCode::Internal("state offsets overflow in transfer"));
}

Prevention

When it happens

Trigger: commit_transferred_state_offsets with a cumulative offset so large that page.state_offsets + offset overflows (u32/u64 depending on build) — typically from an enormous number of transferred aggregate states or corrupted offset bookkeeping.

Common situations: Very large GROUP BY cardinality combined with heavy state transfer across payloads; accounting bugs that double-count offsets.

Related errors

AI-assisted analysis of databendlabs/databend@288d84d76e (2026-09-11). Data as JSON: /api/errors/3dda76ba8f49c3c0. Report an issue: GitHub.

Appendix: source

Thrown at src/query/expression/src/aggregate/payload.rs:104

                .into_iter()
                .map(|(page_index, offset)| (payload_index, page_index, offset)),
        );
    }

    fn commit(self, payloads: &mut [Payload]) {
        for (payload_index, page_index, offset) in self.offsets {
            let payload = payloads
                .get_mut(payload_index)
                .expect("transfer target payload index is out of bounds");

            let page = payload
                .pages
                .get_mut(page_index)
                .expect("transfer target page index is out of bounds");
            let next_state_offsets = page
                .state_offsets
                .checked_add(offset)
                .expect("transfer target state offsets overflow");
            let initialized_states = page
                .rows
                .checked_mul(payload.aggrs.len())
                .expect("transfer target initialized states overflow");

            assert!(
                next_state_offsets <= initialized_states,
                "transfer target state offsets exceed initialized aggregate states"
            );
            page.state_offsets = next_state_offsets;
        }
    }
}

impl<'a> PayloadTransferBatch<'a> {
    pub(super) fn from_flush_state(flush_state: &'a PayloadFlushState) -> Self {
        Self {
            rows: flush_state.row_count,

View on GitHub (pinned to 288d84d76e)