apache/hadoop · error · CommandFormat.UnknownOptionException

Illegal option -t

Error message

Illegal option -t

What it means

MoveFromLocal.processOptions (MoveCommands.java:56) explicitly rejects the -t (thread count) option even though the sibling 'put' command supports it. It throws CommandFormat.UnknownOptionException('-t'), rendered as 'Illegal option -t'. The command's own DESCRIPTION states '-t option has not yet implemented'.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/shell/MoveCommands.java:56

    factory.addClass(Rename.class, "-mv");
  }

  /**
   *  Move local files to a remote filesystem
   */
  public static class MoveFromLocal extends CopyFromLocal {
    public static final String NAME = "moveFromLocal";
    public static final String USAGE =
        "[-f] [-p] [-l] [-d] <localsrc> ... <dst>";
    public static final String DESCRIPTION = 
        "Same as -put, except that the source is " +
        "deleted after it's copied\n" +
        "and -t option has not yet implemented.";

    @Override
    protected void processOptions(LinkedList<String> args) throws IOException {
      if(args.contains("-t")) {
        throw new CommandFormat.UnknownOptionException("-t");
      }
      super.processOptions(args);
    }

    @Override
    protected void processPath(PathData src, PathData target) throws IOException {
      // unlike copy, don't merge existing dirs during move
      if (target.exists && target.stat.isDirectory()) {
        throw new PathExistsException(target.toString());
      }
      super.processPath(src, target);
    }
    
    @Override
    protected void postProcessPath(PathData src) throws IOException {
      if (!src.fs.delete(src.path, false)) {
        // we have no way to know the actual error...
        PathIOException e = new PathIOException(src.toString());

View on GitHub (pinned to 2add963021)

Solutions

1. Drop the -t option from the -moveFromLocal invocation (first and simplest fix)
2. Use 'hadoop fs -put -t <n> <src> <dst>' followed by deleting the local source if you need parallel threads
3. Parallelize externally: split the file list across multiple -moveFromLocal processes or use DistCp for bulk moves

Example fix

# before
hadoop fs -moveFromLocal -t 4 bigfile.log /data/
# -moveFromLocal: Illegal option -t

# after
hadoop fs -moveFromLocal bigfile.log /data/

# or, if threads are required:
hadoop fs -put -t 4 bigfile.log /data/ && rm bigfile.log

// in shell wrappers, whitelist flags per command
if [[ "$cmd" == *moveFromLocal* && "$args" == *" -t"* ]]; then
  echo "-t unsupported for moveFromLocal; stripping"; args=${args/ -t*/};
fi

catch (CommandFormat.UnknownOptionException e) { re-run without the -t flag or delegate to 'put -t' plus explicit source delete }

When it happens

Trigger: Running 'hadoop fs -moveFromLocal -t 4 localsrc dst' (any -t with or without a value). Copying a tuned '-put -t N' command line and swapping only the verb to -moveFromLocal.

Common situations: Scripts migrated from hdfs dfs -put -t <threads> for parallel uploads; version upgrades where -t was added to put (HADOOP-15880 era) and operators assumed parity; CI pipelines parameterizing thread counts across upload commands.

Related errors

• File exists
• No such file or directory
• Input/output error
• Option '-moveToLocal' is not implemented yet.
• Does not match target filesystem

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/7ca4d240755ba747. Report an issue: GitHub.