apache/hadoop · error · IOException

stdin (-) must be the sole input argument when present

Error message

stdin (-) must be the sole input argument when present

What it means

IOException('stdin (-) must be the sole input argument when present') thrown by AppendToFile.processArguments (CopyCommands.java:407). The command noticed readStdin==true (an argument was '-') but additional file arguments were also supplied. appendToFile reads either stdin or a list of files per invocation, never both, so the mix is rejected when the output stream is already open.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/shell/CopyCommands.java:407

      super.processOptions(args);
    }

    @Override
    protected void processArguments(LinkedList<PathData> args)
        throws IOException {

      if (!dst.exists) {
        dst.fs.create(dst.path, false).close();
      }

      InputStream is = null;
      try (FSDataOutputStream fos = appendToNewBlock ?
          dst.fs.append(dst.path, true) : dst.fs.append(dst.path)) {
        if (readStdin) {
          if (args.size() == 0) {
            IOUtils.copyBytes(System.in, fos, DEFAULT_IO_LENGTH);
          } else {
            throw new IOException(
                "stdin (-) must be the sole input argument when present");
          }
        }

        // Read in each input file and write to the target.
        for (PathData source : args) {
          is = Files.newInputStream(source.toFile().toPath());
          IOUtils.copyBytes(is, fos, DEFAULT_IO_LENGTH);
          IOUtils.closeStream(is);
          is = null;
        }
      } finally {
        if (is != null) {
          IOUtils.closeStream(is);
        }
      }
    }
  }

View on GitHub (pinned to 2add963021)

Solutions

1. Use stdin alone: 'cat data | hadoop fs -appendToFile - /dst'
2. Or use files alone: 'hadoop fs -appendToFile a.log b.log /dst' — append the same destination again in a second call to get both inputs
3. Fix the arg-building script so '-' and file names are mutually exclusive branches

Example fix

# before
ARGS="-"
for f in *.log; do ARGS="$ARGS $f"; done
hadoop fs -appendToFile $ARGS /dst     # stdin (-) must be the sole input argument

# after
cat header.txt | hadoop fs -appendToFile - /dst
hadoop fs -appendToFile *.log /dst

boolean hasStdin = args.contains("-");
if (hasStdin && args.size() > 1) {
  throw new IOException("stdin (-) must be the sole input argument when present");
}

try {
  // process append inputs
} catch (IOException e) {
  if (e.getMessage().contains("sole input argument")) {
    // split into two invocations: one for '-', one for the files
  }
}

When it happens

Trigger: 'hadoop fs -appendToFile - extra.txt /dst'; a script conditionally builds an arg list that keeps '-' while also appending file names: ARGS='-'; for f in *.log; do ARGS="$ARGS $f"; done.

Common situations: Pipelines that stream a header from stdin then append files; arg-list builders in bash that seed the list with '-' and forget to drop it when files are added.

Related errors

• missing destination argument
• Illegal option {}
• Not enough arguments: expected {} but got {}
• Too many arguments: expected {} but got {}
• Target path not specified. <target path> <src path> <src pat

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/6af0dac1f54df518. Report an issue: GitHub.