apache/hadoop · error · IOException

Failed to create a URI (URISyntaxException) for the remote p

Error message

Failed to create a URI (URISyntaxException) for the remote path {}. This was based on the archive parameter: {}

What it means

The archives counterpart of the defensive post-copy URI rebuild: after copyRemoteFiles stages the archive, getPathURI(newPath, tmpURI.getFragment()) reassembles the cache URI, and a fragment (the '#alias' of your -archives entry) that is invalid as a URI fragment makes the constructor throw URISyntaxException. It is wrapped in IOException('Failed to create a URI (URISyntaxException) ... based on the archive parameter') even though the comment says it should not throw — so the trigger is a malformed alias in the original entry, not the copy itself.

Source

Thrown at hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/JobResourceUploader.java:397

        }
        Path tmp = new Path(tmpURI);
        URI newURI = null;
        boolean uploadToSharedCache = false;
        if (scConfig.isSharedCacheArchivesEnabled()) {
          newURI = useSharedCache(tmpURI, tmp.getName(), statCache, conf, true);
          if (newURI == null) {
            uploadToSharedCache = true;
          }
        }

        if (newURI == null) {
          Path newPath =
              copyRemoteFiles(archivesDir, tmp, conf, submitReplication);
          try {
            newURI = getPathURI(newPath, tmpURI.getFragment());
          } catch (URISyntaxException ue) {
            // should not throw a uri exception
            throw new IOException(
                "Failed to create a URI (URISyntaxException) for the"
                    + " remote path " + newPath
                    + ". This was based on the archive parameter: "
                    + tmpArchives,
                ue);
          }
        }

        job.addCacheArchive(newURI);
        if (scConfig.isSharedCacheArchivesEnabled()) {
          archiveSCUploadPolicies.put(newURI.toString(), uploadToSharedCache);
        }
      }
    }
  }

  @VisibleForTesting
  void uploadJobJar(Job job, String jobJar, Path submitJobDir,

View on GitHub (pinned to 2add963021)

Solutions

1. Restrict aliases to identifier characters: 'models.tar.gz#models'
2. Encode spaces as %20 if you must keep them
3. Skip the fragment when the archive's own name is acceptable as the link name
4. Validate the fragment separately: new URI(null, null, null, "...") before submission

Example fix

# before
hadoop jar app.jar Driver -archives "hdfs://nn/b/models.tar.gz#model dir" in out
# IOException: Failed to create a URI (URISyntaxException) ... archive parameter

# after
hadoop jar app.jar Driver -archives "hdfs://nn/b/models.tar.gz#models" in out

// reject archive entries with unsafe '#alias' fragments
for (String arc : StringUtils.getStrings(conf.get("tmparchives"))) {
  int i = arc.indexOf('#');
  if (i >= 0) {
    try { new URI(null, null, null, arc.substring(i + 1)); }
    catch (URISyntaxException e) {
      throw new IllegalArgumentException("Bad alias in -archives entry: " + arc, e);
    }
  }
}

static boolean hasSafeFragment(String entry) {
  int i = entry.indexOf('#');
  if (i < 0) return true;
  try { new URI(null, null, null, entry.substring(i + 1)); return true; }
  catch (URISyntaxException e) { return false; }
}

When it happens

Trigger: -archives 'hdfs://nn/b/models.tar.gz#model dir' (space in the alias); fragments with '?', backslash, or quote characters; aliases produced by unescaped variable expansion in scripts.

Common situations: Giving archives readable link names for task-side extraction directories; templated Oozie/shell command lines interpolating version or env names into the fragment.

Related errors

• Resource name must be relative
• Invalid path URI: {} - cannot contain both a URI fragment an
• Invalid path URI: {} - cannot contain both a URI fragment an
• Failed to create a URI (URISyntaxException) for the remote p
• Failed to create a URI (URISyntaxException) for the remote p

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/fbd38d86922ead59. Report an issue: GitHub.