apache/hadoop · error · IllegalArgumentException

Invalid capacity/limit

Error message

Invalid capacity/limit

What it means

BoundedByteArrayOutputStream writes into a fixed byte buffer and refuses construction/reset with a limit larger than the usable capacity, or with negative values. resetBuffer() throws IllegalArgumentException when capacity (buf.length - offset) is less than limit or when either is negative ((capacity | limit) < 0 catches both negative).

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/BoundedByteArrayOutputStream.java:66

  /**
   * Create a BoundedByteArrayOutputStream with the specified
   * capacity and limit.
   * @param capacity The capacity of the underlying byte array
   * @param limit The maximum limit upto which data can be written
   */
  public BoundedByteArrayOutputStream(int capacity, int limit) {
    this(new byte[capacity], 0, limit);
  }

  protected BoundedByteArrayOutputStream(byte[] buf, int offset, int limit) {
    resetBuffer(buf, offset, limit);
  }
  
  protected void resetBuffer(byte[] buf, int offset, int limit) {
    int capacity = buf.length - offset;
    if ((capacity < limit) || (capacity | limit) < 0) {
      throw new IllegalArgumentException("Invalid capacity/limit");
    }
    this.buffer = buf;
    this.startOffset = offset;
    this.currentPointer = offset;
    this.limit = offset + limit;
  }
  
  @Override
  public void write(int b) throws IOException {
    if (currentPointer >= limit) {
      throw new EOFException("Reaching the limit of the buffer.");
    }
    buffer[currentPointer++] = (byte) b;
  }

  @Override
  public void write(byte b[], int off, int len) throws IOException {
    if ((off < 0) || (off > b.length) || (len < 0) || ((off + len) > b.length)

View on GitHub (pinned to 2add963021)

Solutions

1. Ensure limit <= capacity: allocate new byte[limit] if you need the full limit, i.e. new BoundedByteArrayOutputStream(limit, limit).
2. Validate computed sizes against zero before constructing and clamp or fail with context (which config key was bad).
3. When reusing a buffer with an offset, recompute capacity as buf.length - offset and size the limit within it.

Example fix

// before
int limit = conf.getInt("my.limit", 64 * 1024);
new BoundedByteArrayOutputStream(16 * 1024, limit); // throws when limit > 16K

// after
int limit = Math.min(conf.getInt("my.limit", 64 * 1024), 16 * 1024);
new BoundedByteArrayOutputStream(16 * 1024, limit);

int limit = Math.min(configuredLimit, capacity);
if (capacity <= 0 || limit < 0) {
  throw new IllegalArgumentException("capacity=" + capacity + ", limit=" + limit);
}
new BoundedByteArrayOutputStream(capacity, limit);

When it happens

Trigger: new BoundedByteArrayOutputStream(10, 20) — capacity 10 with limit 20; passing a negative limit or offset such that usable capacity is negative; subclass constructors forwarding bad (buf, offset, limit) triples.

Common situations: Sizing shuffle/merge buffers (e.g. mapreduce task output buffers) from configuration where limit and capacity are computed independently; passing an offset larger than the buffer length when reusing a shared buffer.

Related errors

• Limit exceeds buffer size
• Cannot seek to a negative offset
• Requested more bytes than destination buffer size: request l
• write (b[{b.length}], {off}, {len})
• Reaching the limit of the buffer.

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/8cf94e8e624ecc6f. Report an issue: GitHub.