apache/hadoop · error · HadoopIllegalArgumentException

Memory " + maxMemory + " must be greater than or equal to 0

Error message

Memory " + maxMemory + " must be greater than or equal to 0

What it means

LightWeightGSet is the hash structure Hadoop uses for large NameNode maps (BlocksMap, INodeMap, CacheManager, RetryCache). computeCapacity() turns a percentage of a memory budget into an entry-table size. Before computing, it validates both inputs: percentage must be in [0.0, 100.0] and maxMemory must be >= 0; a negative maxMemory throws HadoopIllegalArgumentException. All production call sites use the public overload computeCapacity(percentage, mapName) which feeds Runtime.getRuntime().maxMemory() (always positive), so this throw comes from direct calls to the package-visible (long, double, String) overload or tests, as TestGSet does with computeCapacity(-1, 50.0, "testMap").

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/LightWeightGSet.java:388

   * @param mapName mapName.
   * @param percentage percentage.
   * @return compute capacity.
   */
  public static int computeCapacity(double percentage, String mapName) {
    return computeCapacity(Runtime.getRuntime().maxMemory(), percentage,
        mapName);
  }
  
  @VisibleForTesting
  static int computeCapacity(long maxMemory, double percentage,
      String mapName) {
    if (percentage > 100.0 || percentage < 0.0) {
      throw new HadoopIllegalArgumentException("Percentage " + percentage
          + " must be greater than or equal to 0 "
          + " and less than or equal to 100");
    }
    if (maxMemory < 0) {
      throw new HadoopIllegalArgumentException("Memory " + maxMemory
          + " must be greater than or equal to 0");
    }
    if (percentage == 0.0 || maxMemory == 0) {
      return 0;
    }
    //VM detection
    //See http://java.sun.com/docs/hotspot/HotSpotFAQ.html#64bit_detection
    final String vmBit = System.getProperty("sun.arch.data.model");

    //Percentage of max memory
    final double percentDivisor = 100.0/percentage;
    final double percentMemory = maxMemory/percentDivisor;
    
    //compute capacity
    final int e1 = (int)(Math.log(percentMemory)/Math.log(2.0) + 0.5);
    final int e2 = e1 - ("32".equals(vmBit)? 2: 3);
    final int exponent = e2 < 0? 0: e2 > 30? 30: e2;
    final int c = 1 << exponent;

View on GitHub (pinned to 2add963021)

Solutions

1. Pass a non-negative memory value, typically Runtime.getRuntime().maxMemory(), or use the public computeCapacity(percentage, mapName) overload
2. If the value comes from configuration, validate it at parse time and substitute a sane default instead of forwarding -1
3. Keep percentage within [0.0, 100.0]; note that 0.0 percentage or 0 maxMemory legally yields capacity 0

Example fix

// before
long mem = Long.parseLong(conf.get("mymap.maxmemory", "-1"));
int cap = LightWeightGSet.computeCapacity(mem, 25.0, "mymap");

// after
long mem = conf.getLong("mymap.maxmemory", Runtime.getRuntime().maxMemory());
int cap = LightWeightGSet.computeCapacity(Math.max(0L, mem), 25.0, "mymap");

long maxMemory = Runtime.getRuntime().maxMemory(); // or config-sourced
double pct = 25.0;
if (maxMemory < 0) throw new IllegalArgumentException("maxMemory < 0: " + maxMemory);
if (pct < 0.0 || pct > 100.0) throw new IllegalArgumentException("pct out of range: " + pct);
int capacity = LightWeightGSet.computeCapacity(maxMemory, pct, "myMap");

try { cap = LightWeightGSet.computeCapacity(mem, pct, "myMap"); } catch (HadoopIllegalArgumentException e) { throw new IllegalArgumentException("Bad capacity config for myMap: " + e.getMessage(), e); }

When it happens

Trigger: Calling LightWeightGSet.computeCapacity(maxMemory, percentage, mapName) directly with maxMemory < 0; a custom GSet-sizing wrapper that parses a memory value from configuration (unset property defaulting to a -1 sentinel) and forwards it without a bounds check; unit tests enumerating invalid arguments.

Common situations: Writing unit tests for NameNode-side GSet sizing; porting the computeCapacity idiom into your own component with a config-sourced memory value; practically unreachable in stock Hadoop daemons because they pass Runtime.maxMemory().

Understand the failure class

Background: "Invalid value" and "allowed values are" config errors: what your library rejected and how to fix it — this error's family across 41 libraries.

Related errors

• Illegal initial capacity: " + initCapacity
• Illegal option {}
• Not enough arguments: expected {} but got {}
• Too many arguments: expected {} but got {}
• Target path not specified. <target path> <src path> <src pat

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/f507417eedba3f80. Report an issue: GitHub.