apache/hadoop · error · ArithmeticException

Overflow: for n = {n}, the least power of two (the least int

Error message

Overflow: for n = {n}, the least power of two (the least integer x with x >= n and x a power of two) = {overflow} > Integer.MAX_VALUE = 2147483647

What it means

leastPowerOfTwo rounds up via highestOneBit(n) << 1; for any n in (2^30, Integer.MAX_VALUE] the true answer is 2^31, which does not fit in an int. The helper detects the wrap (roundUp < 0), recomputes the value as a long, and throws ArithmeticException explaining that the least power of two exceeds Integer.MAX_VALUE. This protects callers from silently receiving a negative bucket size.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/util/ByteArrayManager.java:75

   * @return the least power of two greater than or equal to n, i.e. return
   *         the least integer x with x >= n and x a power of two.
   *
   * @throws HadoopIllegalArgumentException
   *           if n <= 0.
   */
  public static int leastPowerOfTwo(final int n) {
    if (n <= 0) {
      throw new HadoopIllegalArgumentException("n = " + n + " <= 0");
    }

    final int highestOne = Integer.highestOneBit(n);
    if (highestOne == n) {
      return n; // n is a power of two.
    }
    final int roundUp = highestOne << 1;
    if (roundUp < 0) {
      final long overflow = ((long) highestOne) << 1;
      throw new ArithmeticException(
          "Overflow: for n = " + n + ", the least power of two (the least"
          + " integer x with x >= n and x a power of two) = "
          + overflow + " > Integer.MAX_VALUE = " + Integer.MAX_VALUE);
    }
    return roundUp;
  }

  /**
   * A counter with a time stamp so that it is reset automatically
   * if there is no increment for the time period.
   */
  static class Counter {
    private final long countResetTimePeriodMs;
    private long count = 0L;
    private long timestamp = Time.monotonicNow();

    Counter(long countResetTimePeriodMs) {
      this.countResetTimePeriodMs = countResetTimePeriodMs;

View on GitHub (pinned to 2add963021)

Solutions

1. Cap the requested size at 1 GiB or reject larger requests before rounding
2. Do sizing math in long and validate it fits an int earlier in the pipeline
3. If multi-GB buffers are legitimate, allocate them directly (new byte[]) instead of through the power-of-two bucket scheme

Example fix

// before
int bucket = ByteArrayManager.leastPowerOfTwo(requestedBytes); // 2_000_000_000 -> throws

// after
if (requestedBytes > (1 << 30)) {
  throw new IllegalArgumentException("buffer too large for pooling: " + requestedBytes);
}
int bucket = ByteArrayManager.leastPowerOfTwo(requestedBytes);

if (n > (1 << 30)) {
  throw new IllegalArgumentException("size too large for power-of-two bucketing: " + n);
}
int rounded = ByteArrayManager.leastPowerOfTwo(n);

try {
  return ByteArrayManager.leastPowerOfTwo(n);
} catch (ArithmeticException e) {
  // the rounded size exceeds Integer.MAX_VALUE; reject or switch to direct allocation
  throw new IllegalArgumentException("buffer too large: " + n, e);
}

When it happens

Trigger: leastPowerOfTwo(n) with n > 1073741824 - requesting a pooled byte-buffer bucket for a roughly 2 GB allocation, or passing a size that grew past 1 GiB through int arithmetic.

Common situations: Applications wiring user-provided buffer sizes into ByteArrayManager-backed paths; stress tests with multi-gigabyte chunks; int multiplication overflowing before the call.

Related errors

• n = {n} <= 0
• Illagal value: ${NAME} = ${value} < MIN = ${MIN}
• ${className} does not support positioned read.
• ${className} does not support positioned reads with byte buf
• ${className} does not support positioned readFully.

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/d3b5cc087a4f33a2. Report an issue: GitHub.