apache/hadoop · error · IndexOutOfBoundsException

write (b[{b.length}], {off}, {len})

Error message

write (b[{b.length}], {off}, {len})

What it means

DataBlocks.validateWriteArgs enforces the standard OutputStream.write(b, off, len) contract for the block-based buffering used by filesystem upload paths (e.g. S3A fast upload): a null buffer throws NullPointerException; negative off/len, off beyond the buffer, or off + len exceeding b.length (including int overflow, checked via (off + len) < 0) throws IndexOutOfBoundsException('write (b[<len>], <off>, <len>)').

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/store/DataBlocks.java:115

  }

  /**
   * Validate args to a write command. These are the same validation checks
   * expected for any implementation of {@code OutputStream.write()}.
   *
   * @param b   byte array containing data.
   * @param off offset in array where to start.
   * @param len number of bytes to be written.
   * @throws NullPointerException      for a null buffer
   * @throws IndexOutOfBoundsException if indices are out of range
   * @throws IOException raised on errors performing I/O.
   */
  public static void validateWriteArgs(byte[] b, int off, int len)
      throws IOException {
    Preconditions.checkNotNull(b);
    if ((off < 0) || (off > b.length) || (len < 0) ||
        ((off + len) > b.length) || ((off + len) < 0)) {
      throw new IndexOutOfBoundsException(
          "write (b[" + b.length + "], " + off + ", " + len + ')');
    }
  }

  /**
   * Create a factory.
   *
   * @param keyToBufferDir Key to buffer directory config for a FS.
   * @param configuration  factory configurations.
   * @param name           factory name -the option from {@link CommonConfigurationKeys}.
   * @return the factory, ready to be initialized.
   * @throws IllegalArgumentException if the name is unknown.
   */
  public static BlockFactory createFactory(String keyToBufferDir,
      Configuration configuration,
      String name) {
    LOG.debug("Creating DataFactory of type : {}", name);
    switch (name) {

View on GitHub (pinned to 2add963021)

Solutions

1. Fix the arithmetic so 0 <= off <= off + len <= b.length
2. Clamp the delegated length to b.length - off
3. Compute off + len in long when either value can approach Integer.MAX_VALUE

Example fix

// before
block.write(data, offset, data.length); // off+len can exceed length

// after
block.write(data, offset, data.length - offset);

static boolean validWriteBounds(byte[] b, int off, int len) {
  return b != null && off >= 0 && len >= 0
      && (long) off + (long) len <= b.length;
}

Catch IndexOutOfBoundsException (message 'write (b[...], off, len)') from block writes; it identifies argument math errors — fix the caller's off/len computation instead of retrying.

When it happens

Trigger: Block write calls with off + len > b.length; negative off or len; (off + len) wrapping past Integer.MAX_VALUE; passing length = b.length while also passing a non-zero offset.

Common situations: Custom OutputStreams delegating with adjusted indices; loops that advance an offset but keep reusing the full array length; huge buffers triggering int overflow.

Related errors

• Requested more bytes than destination buffer size: request l
• hflush not supported by {out}
• hsync not supported by {out}
• Unsupported block buffer "{name}"
• Invalid capacity/limit

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/456436fd60626722. Report an issue: GitHub.