apache/hadoop · error · IOException

unexpected URISyntaxException

Error message

unexpected URISyntaxException

What it means

IOException('unexpected URISyntaxException') thrown by Getmerge.processOptions (CopyCommands.java:87). The destination argument (args.removeLast()) is parsed with 'new URI(...)' and fails because it contains characters java.net.URI rejects (spaces, unescaped %, brackets, etc.). The raw URISyntaxException is chained as the cause for the real parse position.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/shell/CopyCommands.java:87

    protected List<PathData> srcs = null;

    @Override
    protected void processOptions(LinkedList<String> args) throws IOException {
      try {
        CommandFormat cf = new CommandFormat(2, Integer.MAX_VALUE, "nl",
            "skip-empty-file");
        cf.parse(args);

        delimiter = cf.getOpt("nl") ? "\n" : null;
        skipEmptyFileDelimiter = cf.getOpt("skip-empty-file");

        dst = new PathData(new URI(args.removeLast()), getConf());
        if (dst.exists && dst.stat.isDirectory()) {
          throw new PathIsDirectoryException(dst.toString());
        }
        srcs = new LinkedList<PathData>();
      } catch (URISyntaxException e) {
        throw new IOException("unexpected URISyntaxException", e);
      }
    }

    @Override
    protected void processArguments(LinkedList<PathData> items)
    throws IOException {
      super.processArguments(items);
      if (exitCode != 0) { // check for error collecting paths
        return;
      }
      FSDataOutputStream out = dst.fs.create(dst.path);
      try {
        for (PathData src : srcs) {
          if (src.stat.getLen() != 0) {
            // Always do sequential reads.
            try (FSDataInputStream in = src.openForSequentialIO()) {
              IOUtils.copyBytes(in, out, getConf(), false);
              writeDelimiter(out);

View on GitHub (pinned to 2add963021)

Solutions

1. Quote the argument and remove illegal characters: 'hadoop fs -getmerge /src "/local/merged file.txt"' will still fail — pick a name without spaces
2. Percent-encode reserved characters (space as %20) so 'new URI(String)' accepts the destination
3. Simplify the destination to letters, digits, dash, underscore, dot, slash
4. Print the destination variable just before the call in the failing script to see what actually expanded

Example fix

# before
DEST="merged data.txt"
hadoop fs -getmerge /src/dir $DEST        # unexpected URISyntaxException

# after
DEST="merged_data.txt"
hadoop fs -getmerge /src/dir "$DEST"

try {
  new URI(dstArg);
} catch (URISyntaxException e) {
  throw new IllegalArgumentException("Destination is not a valid URI: " + dstArg, e);
}

static boolean isParsableUri(String arg) {
  try { new URI(arg); return true; } catch (URISyntaxException e) { return false; }
}

try {
  dst = new PathData(new URI(args.removeLast()), getConf());
} catch (URISyntaxException e) {
  throw new IOException("unexpected URISyntaxException", e);
}

When it happens

Trigger: 'hadoop fs -getmerge /src /out put.txt' (unquoted space); a destination containing '[', ']', '|', or a bare '%' that is not a valid percent-escape; shell variables that expand to empty or to text with spaces.

Common situations: Unquoted shell arguments in scripts; copy/pasted Windows-style paths (C:\dir with backslashes); templated destinations where a variable expands with whitespace; the same script works for simple names and breaks for filenames with spaces.

Related errors

• Unexpected URISyntaxException: ${e}
• Is a directory
• GCS path supports only '%s' scheme, instead got '%s' from '%
• Bad configuration of hadoop.security.key.provider.path at ${
• Uri without authority: {uri}

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/e311e2d96831a2e6. Report an issue: GitHub.