apache/druid · error · org.apache.druid.java.util.common.IAE

pos out of range [ , ]

Error message

pos %d out of range [%d, %d]

What it means

ByteBufferWriteOutBytes.readFully validates that the requested absolute read position lies within the written data [0, size] before copying bytes into the caller's buffer. It throws IAE when pos is negative or beyond the current size, preventing reads past what has actually been written.

Solutions

1. Verify the pos argument is within [0, size] before calling readFully; recompute or reload offsets from the current writer.
2. Ensure all writes are complete/flushed (writeIntsealed/write calls done) before reading back; call size() to confirm expected length.
3. Check for stale or corrupted offset metadata (smile/json index files) and regenerate the segment if offsets exceed size.
4. Guard against overflow with Ints.checkedCast semantics: confirm pos fits the in-memory buffer sizing model.

Example fix

// before
outBytes.readFully(offset, buffer); // offset may exceed size after rewrite
// after
if (offset < 0 || offset > outBytes.size()) {
  throw new IllegalStateException("stale offset " + offset + " > size " + outBytes.size());
}
outBytes.readFully(offset, buffer);

if (pos < 0 || pos > writeOutBytes.size()) throw new IllegalArgumentException("pos out of range: " + pos);

boolean validPos(WriteOutBytes b, long pos) { return b != null && pos >= 0 && pos <= b.size(); }

try { out.readFully(pos, buf); } catch (IllegalArgumentException e) { throw new IOException("stale offset for read: " + pos, e); }

When it happens

Trigger: Calling readFully(pos, buffer) with pos < 0, pos > size (e.g. reading before writing finished, stale offset after truncation/rewrite, or offset read from a corrupted header/index).

Common situations: Index/offset metadata stored earlier points past the current data (segment rewritten or partially written); off-by-one using size vs size-1; integer overflow of pos; concurrent writers changed size between computing the offset and reading.

Understand the failure class

Background: "value must be between 0 and 1" / "out of range" / "must not be negative" errors: fixing range-validation failures across open-source libraries — this error's family across 42 libraries.

Related errors

• pos out of range [ , ]
• pos out of range [ , ]
• Invalid arguments for reading
• 'skip' must be greater than zero

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/4a41e97ab28d7436. Report an issue: GitHub.

Appendix: source

Thrown at processing/src/main/java/org/apache/druid/segment/writeout/ByteBufferWriteOutBytes.java:210

   */
  public void writeTo(ByteBuffer out)
  {
    checkOpen();
    for (int i = 0; i <= headBufferIndex; i++) {
      ByteBuffer buffer = buffers.get(i);
      buffer.flip();
      out.put(buffer);
      // switch back to the initial state
      buffer.limit(buffer.capacity());
    }
  }

  @Override
  public void readFully(long pos, ByteBuffer buffer)
  {
    checkOpen();
    if (pos < 0 || pos > size) {
      throw new IAE("pos %d out of range [%d, %d]", pos, 0, size);
    }
    int ourBufferIndex = Ints.checkedCast(pos / BUFFER_SIZE);
    int ourBufferOffset = Ints.checkedCast(pos % BUFFER_SIZE);
    for (int bytesLeft = buffer.remaining(); bytesLeft > 0;) {
      int bytesToWrite = Math.min(BUFFER_SIZE - ourBufferOffset, bytesLeft);
      ByteBuffer ourBuffer = buffers.get(ourBufferIndex);
      int ourBufferPosition = ourBuffer.position();
      if (bytesToWrite > ourBufferPosition - ourBufferOffset) {
        throw new BufferUnderflowException();
      }
      try {
        ourBuffer.position(ourBufferOffset);
        ourBuffer.limit(ourBufferOffset + bytesToWrite);
        buffer.put(ourBuffer);
      }
      finally {
        // switch back to the initial state
        ourBuffer.limit(ourBuffer.capacity());

View on GitHub (pinned to 9b90983fd2)