What causes "transfer target payload index is out of bounds" in databendlabs/databend?

Verify that offsets are recorded and committed against the same, unchanged payloads collection — never drop/compact payloads between transfer and commit.

databendlabs/databend · error

transfer target payload index is out of bounds

Error message

transfer target payload index is out of bounds

What it means

During aggregation state transfer, commit() replays recorded (payload_index, page_index, offset) moves against the destination payloads slice. This panic fires when payloads.get_mut(payload_index) returns None, meaning the recorded target payload index no longer exists in the slice — an internal invariant violation in the aggregate payload arena.

Solutions

Verify that offsets are recorded and committed against the same, unchanged payloads collection — never drop/compact payloads between transfer and commit.
Assert/validate payload_index < payloads.len() at record time to fail earlier with a clearer message.
Update the transfer bookkeeping so it is re-derived after any payload arena mutation.
File a bug with the query shape; this is an internal invariant break, not user-fixable input.

Defensive patterns

Strategy: validation

Validate before calling

assert!(payload_index < payloads.len(), "transfer offset references invalid payload {}", payload_index);

Prevention

Never mutate (drop/compact) the payloads arena between recording and committing transfers
Validate indices at record time
Treat any hit as a bug report with the query plan attached

When it happens

Trigger: Calling commit_transferred_state_offsets when the offsets record a payload_index >= payloads.len(), typically after payloads were compacted/dropped or state transfer bookkeeping was computed against a different payload set.

Common situations: Distributed/window aggregate spilling or partial-transfer code paths where payload arenas are reallocated between move recording and commit; bugs after refactorings of the aggregate payload management.

Related errors

AI-assisted analysis of databendlabs/databend@288d84d76e (2026-09-11). Data as JSON: /api/errors/5e11308b2ea1cd27. Report an issue: GitHub.

Appendix: source

Thrown at src/query/expression/src/aggregate/payload.rs:95

#[derive(Default)]
pub(super) struct PayloadTransferStateOffsets {
    offsets: Vec<(usize, usize, usize)>,
}

impl PayloadTransferStateOffsets {
    fn record(&mut self, payload_index: usize, offsets: Vec<(usize, usize)>) {
        self.offsets.extend(
            offsets
                .into_iter()
                .map(|(page_index, offset)| (payload_index, page_index, offset)),
        );
    }

    fn commit(self, payloads: &mut [Payload]) {
        for (payload_index, page_index, offset) in self.offsets {
            let payload = payloads
                .get_mut(payload_index)
                .expect("transfer target payload index is out of bounds");

            let page = payload
                .pages
                .get_mut(page_index)
                .expect("transfer target page index is out of bounds");
            let next_state_offsets = page
                .state_offsets
                .checked_add(offset)
                .expect("transfer target state offsets overflow");
            let initialized_states = page
                .rows
                .checked_mul(payload.aggrs.len())
                .expect("transfer target initialized states overflow");

            assert!(
                next_state_offsets <= initialized_states,
                "transfer target state offsets exceed initialized aggregate states"
            );

View on GitHub (pinned to 288d84d76e)