apache/hadoop · error · IllegalArgumentException

Neither source file listing nor source paths present

Error message

Neither source file listing nor source paths present

What it means

The mirror case of the listing/paths conflict: no -f option was given AND the only positional argument was the target itself, so there are zero source paths. DistCp has nothing to copy and aborts during argument parsing.

Source

Thrown at hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/OptionsParser.java:282

    //Copy any source paths in the arguments to the list
    for (int index = 0; index < leftOverArgs.length - 1; index++) {
      sourcePaths.add(new Path(leftOverArgs[index].trim()));
    }

    /* If command has source file listing, use it else, fall back on source
       paths in args.  If both are present, throw exception and bail */
    if (command.hasOption(
        DistCpOptionSwitch.SOURCE_FILE_LISTING.getSwitch())) {
      if (!sourcePaths.isEmpty()) {
        throw new IllegalArgumentException("Both source file listing and " +
            "source paths present");
      }
      return new DistCpOptions.Builder(new Path(getVal(command,
          DistCpOptionSwitch.SOURCE_FILE_LISTING.getSwitch())), targetPath);
    } else {
      if (sourcePaths.isEmpty()) {
        throw new IllegalArgumentException("Neither source file listing nor " +
            "source paths present");
      }
      return new DistCpOptions.Builder(sourcePaths, targetPath);
    }
  }

  private static String getVal(CommandLine command, String swtch) {
    if (swtch == null) {
      return null;
    }
    String optionValue = command.getOptionValue(swtch.trim());
    if (optionValue == null) {
      return null;
    } else {
      return optionValue.trim();
    }
  }

View on GitHub (pinned to 2add963021)

Solutions

Add at least one source path before the target: hadoop distcp <options> <src...> <target>.
Or use a file listing: hadoop distcp -f <listfile> <target>.
Echo the expanded command in the wrapper to catch empty source variables.

Example fix

# before: only the target is present
hadoop distcp hdfs://nn/tgt

# after
hadoop distcp hdfs://nn/src hdfs://nn/tgt

Defensive patterns

Strategy: validation

Validate before calling

// Wrapper validation: without -f you need >= 2 positional args (source(s) + target)
if (!hasF && positional.size() < 2) {
  throw new IllegalArgumentException(
      "No sources given; usage: distcp <opts> <src...> <target> or -f <file> <target>");
}

Try / catch

try {
  OptionsParser.parse(args);
} catch (IllegalArgumentException e) {
  if ("Neither source file listing nor source paths present".equals(e.getMessage())) {
    throw new IllegalArgumentException("Add source path(s) or a -f listing", e);
  }
  throw e;
}

Prevention

Sanity-check that the command has at least two positional paths (or one plus -f).
Use shell nounset mode so an empty $SRC variable fails before distcp runs.

When it happens

Trigger: hadoop distcp hdfs://nn/tgt (single positional argument becomes the target, leaving no sources); hadoop distcp -update hdfs://nn/tgt.

Common situations: forgetting the source entirely; a script whose $SRC expanded to nothing so only the target token survived; operators reversing argument order confusion.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/73a435f4779b9abd. Report an issue: GitHub.