apache/hadoop · error · IllegalArgumentException

Unable to parse arguments. {args}

Error message

Unable to parse arguments. {args}

What it means

The Apache Commons-CLI parse of the distcp command line failed and is rethrown as IllegalArgumentException with the ParseException chained as the cause. Common causes: an unrecognized option, an option that requires a value but got none, or an argument that starts with '-' being consumed as a new option. The inner CustomParser only rewrites a bare -p to the default preserve set, so anything else unexpected is a hard parse failure.

Source

Thrown at hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/OptionsParser.java:93

  /**
   * The parse method parses the command-line options, and creates
   * a corresponding Options object.
   * @param args Command-line arguments (excluding the options consumed
   *              by the GenericOptionsParser).
   * @return The Options object, corresponding to the specified command-line.
   * @throws IllegalArgumentException Thrown if the parse fails.
   */
  public static DistCpOptions parse(String[] args)
      throws IllegalArgumentException {

    CommandLineParser parser = new CustomParser();

    CommandLine command;
    try {
      command = parser.parse(cliOptions, args, true);
    } catch (ParseException e) {
      throw new IllegalArgumentException("Unable to parse arguments. " +
        Arrays.toString(args), e);
    }

    DistCpOptions.Builder builder = parseSourceAndTargetPaths(command);
    builder
        .withAtomicCommit(
            command.hasOption(DistCpOptionSwitch.ATOMIC_COMMIT.getSwitch()))
        .withSyncFolder(
            command.hasOption(DistCpOptionSwitch.SYNC_FOLDERS.getSwitch()))
        .withDeleteMissing(
            command.hasOption(DistCpOptionSwitch.DELETE_MISSING.getSwitch()))
        .withIgnoreFailures(
            command.hasOption(DistCpOptionSwitch.IGNORE_FAILURES.getSwitch()))
        .withOverwrite(
            command.hasOption(DistCpOptionSwitch.OVERWRITE.getSwitch()))
        .withAppend(
            command.hasOption(DistCpOptionSwitch.APPEND.getSwitch()))
        .withSkipCRC(

View on GitHub (pinned to 2add963021)

Solutions

1. Print supported usage with 'hadoop distcp -h' and fix the offending switch (typo or unsupported long form is the most common).
2. Read the chained cause in the stack trace - commons-cli names the exact problem ('Unrecognized option: -x', 'Missing argument for option: m').
3. Quote every argument containing spaces or glob characters so the shell delivers it intact.
4. If a value must start with '-', restructure the value or drop the dash - commons-cli cannot be told otherwise here.

Example fix

# before: typo 'updat' -> Unrecognized option, wrapped as Unable to parse arguments
hadoop distcp -updat -append hdfs://nn/src hdfs://nn/tgt

# after
hadoop distcp -update -append hdfs://nn/src hdfs://nn/tgt

// Programmatic callers: dry-run the parse before submitting anything
try {
  DistCpOptions opts = OptionsParser.parse(args);
} catch (IllegalArgumentException e) {
  // e.getCause() is the ParseException naming the exact bad token
  System.err.println("Bad distcp arguments: " + e.getCause().getMessage());
  throw e;
}

try {
  ToolRunner.run(new DistCp(conf, opts), args);
} catch (IllegalArgumentException e) {
  Throwable cause = e.getCause();
  if (cause instanceof ParseException) {
    // argument error: report token + usage and exit; retrying cannot help
    System.err.println("Parse failed: " + cause.getMessage());
  } else {
    throw e;
  }
}

When it happens

Trigger: Misspelled or unknown switch, e.g. -updat or a long form like --bandwidth that is not registered; -m or -p with a missing value; an option value that itself begins with '-' (e.g. -blocksPerChunk -5) so commons-cli treats it as the next option; shell globbing splitting one argument into stray tokens.

Common situations: typos in switches; copying commands from a different Hadoop version whose switches differ; unquoted arguments containing spaces or glob characters; negative or dash-prefixed values; scripts concatenating unset variables producing '-'.

Understand the failure class

• Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

Related errors

• Bandwidth specified is invalid: {value}
• Number of liststatus threads is invalid: {value}
• Number of maps is invalid: {value}
• blocksPerChunk is invalid: {chunkSizeStr}
• copyBufferSize is invalid: {copyBufferSizeStr}

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/024cb719f5059f4b. Report an issue: GitHub.