apache/flink · error · IllegalStateException

this memory segment has been freed.

Error message

this memory segment has been freed.

What it means

MemorySegment.swapBytes(seg2, tempBuffer, offset1, offset2, len) swaps two memory regions via a temp buffer. After the bounds check fails it checks liveness: if this segment was already freed (address > addressLimit) it throws IllegalStateException('this memory segment has been freed.') — the error identifies the receiver ('this'), not the argument.

Source

Thrown at flink-core/src/main/java/org/apache/flink/core/memory/MemorySegment.java:1588

            byte[] tempBuffer, MemorySegment seg2, int offset1, int offset2, int len) {
        if ((offset1 | offset2 | len | (tempBuffer.length - len)) >= 0) {
            final long thisPos = this.address + offset1;
            final long otherPos = seg2.address + offset2;

            if (thisPos <= this.addressLimit - len && otherPos <= seg2.addressLimit - len) {
                // this -> temp buffer
                UNSAFE.copyMemory(
                        this.heapMemory, thisPos, tempBuffer, BYTE_ARRAY_BASE_OFFSET, len);

                // other -> this
                UNSAFE.copyMemory(seg2.heapMemory, otherPos, this.heapMemory, thisPos, len);

                // temp buffer -> other
                UNSAFE.copyMemory(
                        tempBuffer, BYTE_ARRAY_BASE_OFFSET, seg2.heapMemory, otherPos, len);
                return;
            } else if (this.address > this.addressLimit) {
                throw new IllegalStateException("this memory segment has been freed.");
            } else if (seg2.address > seg2.addressLimit) {
                throw new IllegalStateException("other memory segment has been freed.");
            }
        }

        // index is in fact invalid
        throw new IndexOutOfBoundsException(
                String.format(
                        "offset1=%d, offset2=%d, len=%d, bufferSize=%d, address1=%d, address2=%d",
                        offset1, offset2, len, tempBuffer.length, this.address, seg2.address));
    }

    /**
     * Equals two memory segment regions.
     *
     * @param seg2 Segment to equal this segment with
     * @param offset1 Offset of this segment to start equaling
     * @param offset2 Offset of seg2 to start equaling

View on GitHub (pinned to 2f3c205e92)

Solutions

1. Check this.isFreed() before swapping and abort the sort/spill operation.
2. Fix the lifecycle: do not return segments to the pool until all sort/merge threads using them have completed (drain threads before releasing).
3. If the other segment is the freed one you will see the sibling message 'other memory segment has been freed.' — use the message to tell which side to fix.

Example fix

// before
a.swapBytes(b, tempBuf, off1, off2, len); // 'this ... has been freed.' => a is freed

// after
if (a.isFreed() || b.isFreed()) {
    throw new IllegalStateException("cannot swap: segment already released");
}
a.swapBytes(b, tempBuf, off1, off2, len);

if (seg1.isFreed() || seg2.isFreed()) {
    throw new IllegalStateException("cannot swap: a segment was already freed");
}
seg1.swapBytes(seg2, tempBuf, off1, off2, len);

When it happens

Trigger: Calling swapBytes where the receiving segment has been freed — e.g. in sort/merge code that keeps operating on segments already returned to the memory manager after a spill or cancellation.

Common situations: External sorters reordering buffers after a failed check released them; task cancellation paths that free segments while a sort thread is still swapping; segment pool returned early on exception.

Related errors

• segment has been freed
• other memory segment has been freed.
• MemorySegment can be freed only once!
• The runtime context has not been initialized yet. Try access
• Could not write {numBytes} bytes. Buffer overflow.

AI-assisted analysis of apache/flink@2f3c205e92 (2026-08-14). Data as JSON: /api/errors/925256557661c0b3. Report an issue: GitHub.