apache/hadoop · error · IllegalArgumentException
position out of range: {position} (expected: 0 - {count-1})
Error message
position out of range: {position} (expected: 0 - {count-1}) What it means
Error "position out of range: {position} (expected: 0 - {count-1})" thrown in apache/hadoop.
Source
Thrown at hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-shuffle/src/main/java/org/apache/hadoop/mapred/FadvisedFileRegion.java:112
}
}
/**
* This method transfers data using local buffer. It transfers data from
* a disk to a local buffer in memory, and then it transfers data from the
* buffer to the target. This is used only if transferTo is disallowed in
* the configuration file. super.TransferTo does not perform well on Windows
* due to a small IO request generated. customShuffleTransfer can control
* the size of the IO requests by changing the size of the intermediate
* buffer.
*/
@VisibleForTesting
long customShuffleTransfer(WritableByteChannel target, long position)
throws IOException {
long actualCount = this.count - position;
if (actualCount < 0 || position < 0) {
throw new IllegalArgumentException(
"position out of range: " + position +
" (expected: 0 - " + (this.count - 1) + ')');
}
if (actualCount == 0) {
return 0L;
}
long trans = actualCount;
int readSize;
ByteBuffer byteBuffer = ByteBuffer.allocate(
Math.min(
this.shuffleBufferSize,
trans > Integer.MAX_VALUE ? Integer.MAX_VALUE : (int) trans));
while(trans > 0L &&
(readSize = fileChannel.read(byteBuffer, this.position+position)) > 0) {
//adjust counters and buffer limit
if(readSize < trans) {View on GitHub (pinned to 2add963021)
Solutions
- Request a position within [0, count-1] of the FadvisedFileRegion; fix the caller's offset computation.
- Validate shuffle request parameters so the requested region falls inside the map output file.
When it happens
Trigger: Thrown by FadvisedFileRegion.position() when the requested shuffle position is outside [0, count-1] for the identified map output. Usually a corrupt or stale shuffle request; the reducer should retry the fetch from the correct offset.
Common situations: See trigger scenarios.
AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22).
Data as JSON: /api/errors/9a106764c56bd162.
Report an issue: GitHub.