apache/hadoop · error · IllegalArgumentException

Target path not specified

Error message

Target path not specified

What it means

DistCp takes sources and target from the positional (non-option) arguments; the last positional argument is the target. parseSourceAndTargetPaths found command.getArgs() null or empty, meaning every token was consumed as an option or option value and no target path survived. Note that -f consumes its file path as the option's value - so '-f <file>' with nothing after it leaves zero positional args.

Source

Thrown at hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/OptionsParser.java:259

    return builder.build();
  }

  /**
   * parseSourceAndTargetPaths is a helper method for parsing the source
   * and target paths.
   *
   * @param command command line arguments
   * @return        DistCpOptions
   */
  private static DistCpOptions.Builder parseSourceAndTargetPaths(
      CommandLine command) {
    Path targetPath;
    List<Path> sourcePaths = new ArrayList<Path>();

    String[] leftOverArgs = command.getArgs();
    if (leftOverArgs == null || leftOverArgs.length < 1) {
      throw new IllegalArgumentException("Target path not specified");
    }

    //Last Argument is the target path
    targetPath = new Path(leftOverArgs[leftOverArgs.length - 1].trim());

    //Copy any source paths in the arguments to the list
    for (int index = 0; index < leftOverArgs.length - 1; index++) {
      sourcePaths.add(new Path(leftOverArgs[index].trim()));
    }

    /* If command has source file listing, use it else, fall back on source
       paths in args.  If both are present, throw exception and bail */
    if (command.hasOption(
        DistCpOptionSwitch.SOURCE_FILE_LISTING.getSwitch())) {
      if (!sourcePaths.isEmpty()) {
        throw new IllegalArgumentException("Both source file listing and " +
            "source paths present");
      }

View on GitHub (pinned to 2add963021)

Solutions

  1. Append the destination as the last positional argument: hadoop distcp <options> <src...> <target>.
  2. With -f, the target still must follow the file: hadoop distcp -f <listfile> <target>.
  3. Echo the fully expanded command in wrapper scripts to catch unset variables before submission.

Example fix

# before: -f consumes /tmp/list.txt as its value; no positional target remains
hadoop distcp -f /tmp/list.txt

# after
hadoop distcp -f /tmp/list.txt hdfs://nn/tgt
Defensive patterns

Strategy: validation

Validate before calling

// Wrapper validation: at least one positional (non-option) argument must remain
// after option parsing; the last one is the target.
int positional = 0;
for (String a : args) {
  if (!a.startsWith("-")) positional++; // approximation; use commons-cli for exactness
}
if (positional < 1) {
  throw new IllegalArgumentException("Target path missing; usage: distcp <opts> <src...> <target>");
}
// shell: [ -n "$TARGET" ] || { echo "TARGET unset"; exit 2; }

Try / catch

try {
  OptionsParser.parse(args);
} catch (IllegalArgumentException e) {
  if ("Target path not specified".equals(e.getMessage())) {
    throw new IllegalArgumentException("Append the destination as the last argument", e);
  }
  throw e;
}

Prevention

When it happens

Trigger: hadoop distcp -update (switches only, no paths); hadoop distcp -f /tmp/list.txt (forgot the trailing target); a wrapper script interpolating an unset $TARGET variable so the token never appears.

Common situations: scripts dropping the target when a variable is empty; forgetting that -f takes its value inline and the target must still be appended; interactive typos truncating the command.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/c1aec45e18abbf6d. Report an issue: GitHub.