What causes "offs({}) + len({}) > buf.length({})." in apache/hadoop?

Compute the length from the actual region: int len = Math.min(requestedLen, buf.length - offs); and skip when <= 0.

apache/hadoop · error · IndexOutOfBoundsException

offs({}) + len({}) > buf.length({}).

Error message

offs({}) + len({}) > buf.length({}).

What it means

Third guard in CBZip2OutputStream.write(byte[], offs, len): if offs + len exceeds buf.length it throws IndexOutOfBoundsException("offs(...) + len(...) > buf.length(...)"). The codec refuses ranges that run past the end of the source array. Note offs and len are already proven non-negative by the earlier checks, so this is purely the 'range exceeds array' case.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/compress/bzip2/CBZip2OutputStream.java:869

  /**
  * Returns the blocksize parameter specified at construction time.
  * @return blocksize.
  */
  public final int getBlockSize() {
    return this.blockSize100k;
  }

  @Override
  public void write(final byte[] buf, int offs, final int len)
      throws IOException {
    if (offs < 0) {
      throw new IndexOutOfBoundsException("offs(" + offs + ") < 0.");
    }
    if (len < 0) {
      throw new IndexOutOfBoundsException("len(" + len + ") < 0.");
    }
    if (offs + len > buf.length) {
      throw new IndexOutOfBoundsException("offs(" + offs + ") + len("
          + len + ") > buf.length(" + buf.length + ").");
    }
    if (this.out == null) {
      throw new IOException("stream closed");
    }

    for (int hi = offs + len; offs < hi;) {
      write0(buf[offs++]);
    }
  }

  private void write0(int b) throws IOException {
    if (this.currentChar != -1) {
      b &= 0xff;
      if (this.currentChar == b) {
        if (++this.runLength > 254) {
          writeRun();
          this.currentChar = -1;

View on GitHub (pinned to 2add963021)

Solutions

Compute the length from the actual region: int len = Math.min(requestedLen, buf.length - offs); and skip when <= 0.
Recompute lengths whenever the underlying buffer is replaced or resized — never cache len across buffer swaps.
Use Objects.checkFromToIndex(offs, offs + len, buf.length) to validate the whole range in one call.
For ByteBuffer-backed writes, derive offs/len from position()/remaining() of the same buffer instance.

Example fix

// before
cbz.write(buf, 10, buf.length); // 10 + buf.length > buf.length

// after
cbz.write(buf, 10, Math.min(buf.length - 10, wantedLen));

Defensive patterns

Strategy: validation

Validate before calling

int safeLen = Math.min(len, buf.length - offs);
if (safeLen > 0) {
  cbz.write(buf, offs, safeLen);
}

Prevention

Recompute lengths whenever the backing buffer is swapped or resized.
For ByteBuffer.array(), derive off/len from that buffer's position/remaining, not the view's.
Use Objects.checkFromToIndex for one-shot range validation.

When it happens

Trigger: write(buf, offs, len) where offs + len > buf.length: reusing a stale len after swapping to a smaller buffer, offsets relative to a substring/subregion without adjusting, or off-by-one loops passing len == buf.length while offs > 0.

Common situations: Buffer reuse across iterations (buffer shrinks, length not recomputed); passing (fullArrayLength) as len when starting at a non-zero offset; views over slices (ByteBuffer.array()) where the array length differs from the view's limit; ported C code using pointer + length pairs.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/538c0baf00a44768. Report an issue: GitHub.