apache/hadoop · error · IllegalArgumentException

Bandwidth specified is invalid: {value}

Error message

Bandwidth specified is invalid: {value}

What it means

The -bandwidth option value must parse as a java Float (it is a per-map cap in MB/s). Float.parseFloat threw NumberFormatException and OptionsParser rethrows IllegalArgumentException carrying the raw value. Any unit suffix, thousands separator, or locale-format decimal will fail - only a plain dot-decimal number is accepted.

Source

Thrown at hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/OptionsParser.java:167

          DistCpOptionSwitch.WORK_PATH.getSwitch());
      if (workPath != null && !workPath.isEmpty()) {
        builder.withAtomicWorkPath(new Path(workPath));
      }
    }
    if (command.hasOption(DistCpOptionSwitch.TRACK_MISSING.getSwitch())) {
      builder.withTrackMissing(
          new Path(getVal(
              command,
              DistCpOptionSwitch.TRACK_MISSING.getSwitch())));
    }

    if (command.hasOption(DistCpOptionSwitch.BANDWIDTH.getSwitch())) {
      try {
        final Float mapBandwidth = Float.parseFloat(
            getVal(command, DistCpOptionSwitch.BANDWIDTH.getSwitch()));
        builder.withMapBandwidth(mapBandwidth);
      } catch (NumberFormatException e) {
        throw new IllegalArgumentException("Bandwidth specified is invalid: " +
            getVal(command, DistCpOptionSwitch.BANDWIDTH.getSwitch()), e);
      }
    }

    if (command.hasOption(
        DistCpOptionSwitch.NUM_LISTSTATUS_THREADS.getSwitch())) {
      try {
        final Integer numThreads = Integer.parseInt(getVal(command,
            DistCpOptionSwitch.NUM_LISTSTATUS_THREADS.getSwitch()));
        builder.withNumListstatusThreads(numThreads);
      } catch (NumberFormatException e) {
        throw new IllegalArgumentException(
            "Number of liststatus threads is invalid: " + getVal(command,
                DistCpOptionSwitch.NUM_LISTSTATUS_THREADS.getSwitch()), e);
      }
    }

    if (command.hasOption(DistCpOptionSwitch.MAX_MAPS.getSwitch())) {

View on GitHub (pinned to 2add963021)

Solutions

1. Pass a plain unit-less float in megabytes per second, e.g. -bandwidth 12.5.
2. Remove commas/suffixes from the value ('1,000' or '100mb' are invalid).
3. In wrapper scripts, validate: [[ "$BW" =~ ^[0-9]+([.][0-9]+)?$ ]] || exit with a clear message.
4. Check that the shell variable you interpolate is actually set before invoking distcp.

Example fix

# before
hadoop distcp -bandwidth 100MB hdfs://nn/src hdfs://nn/tgt

# after: plain float, MB per second per map
hadoop distcp -bandwidth 100 hdfs://nn/src hdfs://nn/tgt

// Wrapper validation: -bandwidth must be a plain dot-decimal float (MB/s)
private static void checkBandwidth(String v) {
  if (!v.matches("[0-9]+(\.[0-9]+)?")) {
    throw new IllegalArgumentException(
        "-bandwidth must be a plain number in MB/s, got: " + v);
  }
}
# shell: [[ "$BW" =~ ^[0-9]+([.][0-9]+)?$ ]] || { echo "bad -bandwidth: $BW"; exit 2; }

try {
  OptionsParser.parse(args);
} catch (IllegalArgumentException e) {
  if (e.getMessage().startsWith("Bandwidth specified is invalid")) {
    // strip units/separators from the value and re-parse before retrying
    fixAndRetry(args, "-bandwidth");
  } else {
    throw e;
  }
}

When it happens

Trigger: -bandwidth 100MB (unit suffix), -bandwidth 1,5 (comma decimal), -bandwidth 1e (malformed exponent), or an empty value from an unset shell variable.

Common situations: operators assuming units like 'm' or 'MB' are accepted; non-EN locales typing comma decimals; scripts passing $BW with BW unset; confusion between megabits and megabytes.

Related errors

• Number of liststatus threads is invalid: {value}
• Number of maps is invalid: {value}
• blocksPerChunk is invalid: {chunkSizeStr}
• copyBufferSize is invalid: {copyBufferSizeStr}
• Unable to parse arguments. {args}

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/f74d7b92c6d10a26. Report an issue: GitHub.