What causes "write (b[{}], {}, {})" in apache/hadoop?

Fix the caller's arithmetic: assert 0 <= off && off + len <= b.length before calling write; treat read()==-1 as EOS instead of an offset.

apache/hadoop · error · IndexOutOfBoundsException

write (b[{}], {}, {})

Error message

write (b[{}], {}, {})

What it means

OBSDataBlocks.validateWriteArgs(b, off, len) is the bounds check called at the top of OBSBlockOutputStream.write and elsewhere: after Preconditions.checkNotNull(b), it rejects off<0, len<0, off>b.length, off+len>b.length, or integer-overflow off+len<0 with IndexOutOfBoundsException('write (b[<len>], <off>, <len>)'). The message is the full call signature, letting you immediately see which argument is inconsistent with the buffer size. This is a pure caller bug — the buffer slice described does not exist.

Source

Thrown at hadoop-cloud-storage-project/hadoop-huaweicloud/src/main/java/org/apache/hadoop/fs/obs/OBSDataBlocks.java:75

  private OBSDataBlocks() {
  }

  /**
   * Validate args to a write command. These are the same validation checks
   * expected for any implementation of {@code OutputStream.write()}.
   *
   * @param b   byte array containing data
   * @param off offset in array where to start
   * @param len number of bytes to be written
   * @throws NullPointerException      for a null buffer
   * @throws IndexOutOfBoundsException if indices are out of range
   */
  static void validateWriteArgs(final byte[] b, final int off,
      final int len) {
    Preconditions.checkNotNull(b);
    if (off < 0 || off > b.length || len < 0 || off + len > b.length
        || off + len < 0) {
      throw new IndexOutOfBoundsException(
          "write (b[" + b.length + "], " + off + ", " + len + ')');
    }
  }

  /**
   * Create a factory.
   *
   * @param owner factory owner
   * @param name  factory name -the option from {@link OBSConstants}.
   * @return the factory, ready to be initialized.
   * @throws IllegalArgumentException if the name is unknown.
   */
  static BlockFactory createFactory(final OBSFileSystem owner,
      final String name) {
    switch (name) {
    case OBSConstants.FAST_UPLOAD_BUFFER_ARRAY:
      return new ByteArrayBlockFactory(owner);
    case OBSConstants.FAST_UPLOAD_BUFFER_DISK:

View on GitHub (pinned to 2add963021)

Solutions

Fix the caller's arithmetic: assert 0 <= off && off + len <= b.length before calling write; treat read()==-1 as EOS instead of an offset.
Use System.arraycopy semantics or Arrays.copyOfRange(b, off, off+len) when you need a safe slice.
Reset loop offsets at buffer boundaries; prefer 'while ((n = in.read(buf)) != -1) out.write(buf, 0, n);' which is inherently safe.
For chunked writers, compute len as Math.min(chunkSize, remaining) rather than trusting passed-in sizes.

Example fix

// before
int off = in.read(prevBuf);            // can be -1 at EOF
out.write(buf, off, len);              // -> write (b[8192], -1, 4096)

// after
int n;
while ((n = in.read(buf)) != -1) {
  out.write(buf, 0, n);                // offsets always valid
}

Defensive patterns

Strategy: validation

Validate before calling

static void checkWriteArgs(byte[] b, int off, int len) {
  java.util.Objects.requireNonNull(b);
  if (off < 0 || len < 0 || off > b.length || len > b.length - off) {
    throw new IndexOutOfBoundsException(
        "write(b[" + b.length + "], " + off + ", " + len + ")");
  }
}
checkWriteArgs(buf, off, len); out.write(buf, off, len);

Try / catch

try {
  out.write(buf, off, len);
} catch (IndexOutOfBoundsException e) {
  if (String.valueOf(e.getMessage()).startsWith("write (b[")) {
    throw new IllegalArgumentException("bad (off,len) for buffer — caller bug", e);
  }
  throw e;
}

Prevention

Use the canonical copy loop: while ((n = in.read(buf)) != -1) out.write(buf, 0, n).
Never feed a read() return value into offset without checking for -1.
Compute len as Math.min(chunk, b.length - off).
Note the message format 'write (b[len], off, len)' reports buffer LENGTH first — use it to spot the mismatch.

When it happens

Trigger: Passing offset/len computed from a previous, different buffer; len larger than remaining bytes (off+len>b.length) after arithmetic mistakes; negative offset from a read() that returned -1 and was fed into offset; Integer overflow when off+len wraps around (huge len near Integer.MAX_VALUE); reusing a loop variable as offset without resetting it.

Common situations: Custom InputStream/OutputStream adapters translating between stream APIs; compression or crypto wrappers computing chunk offsets; loops like 'off += read(); write(b, off, len)' that forget read() can return -1; copy helpers mixing up (srcPos, length) argument order.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/3797c39d5de24de3. Report an issue: GitHub.