apache/flink · error · IndexOutOfBoundsException

offset1=%d, offset2=%d, len=%d, bufferSize=%d, address1=%d,

Error message

offset1=%d, offset2=%d, len=%d, bufferSize=%d, address1=%d, address2=%d

What it means

MemorySegment.swapBytes(...) throws a formatted IndexOutOfBoundsException('offset1=%d, offset2=%d, len=%d, bufferSize=%d, address1=%d, address2=%d') when the requested regions are out of bounds on either segment and neither segment is freed. The message dumps every input plus both addresses so you can compare offset1+offset2+len against the actual segment limits.

Source

Thrown at flink-core/src/main/java/org/apache/flink/core/memory/MemorySegment.java:1595

                UNSAFE.copyMemory(
                        this.heapMemory, thisPos, tempBuffer, BYTE_ARRAY_BASE_OFFSET, len);

                // other -> this
                UNSAFE.copyMemory(seg2.heapMemory, otherPos, this.heapMemory, thisPos, len);

                // temp buffer -> other
                UNSAFE.copyMemory(
                        tempBuffer, BYTE_ARRAY_BASE_OFFSET, seg2.heapMemory, otherPos, len);
                return;
            } else if (this.address > this.addressLimit) {
                throw new IllegalStateException("this memory segment has been freed.");
            } else if (seg2.address > seg2.addressLimit) {
                throw new IllegalStateException("other memory segment has been freed.");
            }
        }

        // index is in fact invalid
        throw new IndexOutOfBoundsException(
                String.format(
                        "offset1=%d, offset2=%d, len=%d, bufferSize=%d, address1=%d, address2=%d",
                        offset1, offset2, len, tempBuffer.length, this.address, seg2.address));
    }

    /**
     * Equals two memory segment regions.
     *
     * @param seg2 Segment to equal this segment with
     * @param offset1 Offset of this segment to start equaling
     * @param offset2 Offset of seg2 to start equaling
     * @param length Length of the equaled memory region
     * @return true if equal, false otherwise
     */
    public boolean equalTo(MemorySegment seg2, int offset1, int offset2, int length) {
        int i = 0;

        // we assume unaligned accesses are supported.

View on GitHub (pinned to 2f3c205e92)

Solutions

1. Decode the message: verify offset1 + len <= seg1.size() and offset2 + len <= seg2.size(); the mismatch points at which side's index math is wrong.
2. Re-derive len and offsets from the same entry-size constant used when records were written; check for schema/serializer version drift.
3. Add a bounds assert in your sort/swap loop during development to catch the bad entry before the swap.

Example fix

// before
seg1.swapBytes(seg2, tmp, off1, off2, len); // formatted IndexOutOfBoundsException

// after
checkArgument(off1 >= 0 && off1 + len <= seg1.size(), "bad offset1/len");
checkArgument(off2 >= 0 && off2 + len <= seg2.size(), "bad offset2/len");
seg1.swapBytes(seg2, tmp, off1, off2, len);

checkArgument(off1 >= 0 && off1 + len <= seg1.size(), "offset1=%d len=%d size=%d", off1, len, seg1.size());
checkArgument(off2 >= 0 && off2 + len <= seg2.size(), "offset2=%d len=%d size=%d", off2, len, seg2.size());

try {
    seg1.swapBytes(seg2, tempBuf, off1, off2, len);
} catch (IndexOutOfBoundsException e) {
    // message prints offset1/offset2/len/addresses; compare against segment sizes to find the bad side
    throw e;
}

When it happens

Trigger: Calling swapBytes with offset1/offset2/len such that offset+len exceeds either segment's size (or a negative offset), while both segments are still alive — e.g. sort logic using a wrong entry size or misaligned index entries.

Common situations: Normalized-key or index-entry math drifting from the actual record layout after a schema/serializer change; swapping with a len computed for a different record width; off-by-one rollover when an entry spans the segment boundary.

Related errors

• offset=%d, numBytes=%d, address=%d
• Local output sorting does not support type {inputType} yet.
• Tuple position is out of range: {fieldPos}
• Field expression must be equal to '*' or '_' for atomic type
• Could not write {numBytes} bytes. Buffer overflow.

AI-assisted analysis of apache/flink@2f3c205e92 (2026-08-14). Data as JSON: /api/errors/c3b8cf7755f3a476. Report an issue: GitHub.