apache/hadoop · error · UnknownOptionException

Illegal option {}

Error message

Illegal option {}

What it means

Thrown by CommandFormat.parse() (hadoop-common fs.shell) when a command-line token starting with '-' is neither a registered boolean flag nor a registered value-option of the FsShell command being run. UnknownOptionException extends IllegalArgumentException and its message renders as 'Illegal option -X'. Parsing stops immediately, so no path arguments are processed. It only fires when the command was not built with the ignore-unknown-options wildcard.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/shell/CommandFormat.java:130

      if (options.containsKey(opt)) {
        args.remove(pos);
        options.put(opt, Boolean.TRUE);
      } else if (optionsWithValue.containsKey(opt)) {
        args.remove(pos);
        if (pos < args.size() && (args.size() > minPar)
                && !args.get(pos).startsWith("-")) {
          arg = args.get(pos);
          args.remove(pos);
        } else {
          arg = "";
        }
        if (!arg.startsWith("-") || arg.equals("-")) {
          optionsWithValue.put(opt, arg);
        }
      } else if (ignoreUnknownOpts) {
        pos++;
      } else {
        throw new UnknownOptionException(arg);
      }
    }
    int psize = args.size();
    if (psize < minPar) {
      throw new NotEnoughArgumentsException(minPar, psize);
    }
    if (psize > maxPar) {
      throw new TooManyArgumentsException(maxPar, psize);
    }
  }
  
  /** Return if the option is set or not
   * 
   * @param option String representation of an option
   * @return true is the option is set; false otherwise
   */
  public boolean getOpt(String option) {
    return options.containsKey(option) ? options.get(option) : false;

View on GitHub (pinned to 2add963021)

Solutions

1. Run 'hdfs dfs -help <command>' (or hadoop fs -help) and remove or correct the unsupported option
2. For a path that starts with '-', prefix it with './' or insert '--' before it to force end of option processing
3. Verify the option exists in the Hadoop version actually on PATH ('hadoop version') and use the equivalent supported flag for that release
4. If implementing a custom FsShell command, register the flag via the CommandFormat constructor (boolean flags) or addOptionWithValue() so parse() accepts it

Example fix

# before
hdfs dfs -ls -Z /user/me
# after
hdfs dfs -ls /user/me
# (or, for a file literally named '-data':)
hdfs dfs -cat -- -data
hdfs dfs -cat ./-data

// before invoking a shell command programmatically, confirm every dash-token is expected
Set<String> KNOWN = Set.of("f", "p", "l", "d", "t", "q", "crc", "ignoreCrc", "R", "r");
for (String a : args) {
  if (a.startsWith("-") && !a.equals("-") && !a.equals("--")
      && !KNOWN.contains(a.substring(1).split("=")[0])) {
    throw new IllegalArgumentException("Unsupported option " + a + " - check 'hdfs dfs -help'");
  }
}

try {
  cf.parse(argList);
} catch (CommandFormat.UnknownOptionException e) {
  // fail with command usage instead of proceeding with a wrong interpretation
  printUsageAndExit(e.getMessage());
}

When it happens

Trigger: Running e.g. 'hdfs dfs -ls -Z /user' (-Z is not registered for ls), reusing another command's flag (e.g. '-skipTrash' on -cp), a typo like '-r' where only '-R' exists, or passing a source path that itself begins with '-' (e.g. '-myFile') without a './' prefix or a preceding '--' terminator.

Common situations: Scripts written against a different Hadoop version (flags were added/removed across releases, e.g. -t/-q for multithreaded copy only exist in newer 2.x/3.x); copy-pasting option sets between 'hadoop fs' and 'hdfs dfs' invocations; unquoted filenames starting with '-'; custom FsShell commands that forgot to register a new flag in their CommandFormat constructor.

Related errors

• Not enough arguments: expected {} but got {}
• Too many arguments: expected {} but got {}
• Target path not specified. <target path> <src path> <src pat
• The number of source paths is less than 2. <target path> <sr
• No attribute for {}

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/dc607302bfc80840. Report an issue: GitHub.