apache/hadoop · error · IOException

Error parsing local resource path. Path was not able to be c

Error message

Error parsing local resource path. Path was not able to be converted to a URI: {}

What it means

Thrown by YARNRunner.createApplicationResource while building a LocalResource for the job: it constructs a URI of the form qualifiedPath#symlink (the fragment names the localized symlink) and java.net.URI rejects it with URISyntaxException. The IOException includes the offending qualified path. Practically this means the resource path or the symlink fragment contains characters that are illegal in a URI fragment (spaces, '#', unencoded non-ASCII).

Source

Thrown at hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/YARNRunner.java:374

  private LocalResource createApplicationResource(FileContext fs, Path p,
      String fileSymlink, LocalResourceType type, LocalResourceVisibility viz,
      Boolean uploadToSharedCache) throws IOException {
    LocalResource rsrc = recordFactory.newRecordInstance(LocalResource.class);
    FileStatus rsrcStat = fs.getFileStatus(p);
    // We need to be careful when converting from path to URL to add a fragment
    // so that the symlink name when localized will be correct.
    Path qualifiedPath =
        fs.getDefaultFileSystem().resolvePath(rsrcStat.getPath());
    URI uriWithFragment = null;
    boolean useFragment = fileSymlink != null && !fileSymlink.equals("");
    try {
      if (useFragment) {
        uriWithFragment = new URI(qualifiedPath.toUri() + "#" + fileSymlink);
      } else {
        uriWithFragment = qualifiedPath.toUri();
      }
    } catch (URISyntaxException e) {
      throw new IOException(
          "Error parsing local resource path."
              + " Path was not able to be converted to a URI: " + qualifiedPath,
          e);
    }
    rsrc.setResource(URL.fromURI(uriWithFragment));
    rsrc.setSize(rsrcStat.getLen());
    rsrc.setTimestamp(rsrcStat.getModificationTime());
    rsrc.setType(type);
    rsrc.setVisibility(viz);
    rsrc.setShouldBeUploadedToSharedCache(uploadToSharedCache);
    return rsrc;
  }

  private Map<String, LocalResource> setupLocalResources(Configuration jobConf,
      String jobSubmitDir) throws IOException {
    Map<String, LocalResource> localResources = new HashMap<>();

    Path jobConfPath = new Path(jobSubmitDir, MRJobConfig.JOB_CONF_FILE);

View on GitHub (pinned to 2add963021)

Solutions

1. Rename the resource or symlink so it contains only URI-safe characters (alnum, '-', '_', '.', '~'); avoid spaces and '#' in the fragment
2. URL-encode the fragment when building the name programmatically (URLEncoder.encode(name, "UTF-8"))
3. Sanitize user-supplied filenames before passing them to -libjars/-archives/-files
4. Verify with a quick new URI(path + "#" + symlink) smoke test on the client

Example fix

# before
hadoop jar app.jar -libjars "hdfs:/share/my lib.jar#my lib.jar" MyDriver

# after
hadoop jar app.jar -libjars "hdfs:/share/my_lib.jar#mylib.jar" MyDriver
# no spaces in path or fragment; fragment is the localized symlink name

static String safeFragment(String symlink) {
  String s = symlink == null ? "" : symlink.trim();
  if (!s.matches("[A-Za-z0-9._~-]+"))
    throw new IllegalArgumentException("symlink has URI-unsafe characters: " + s);
  return s.isEmpty() ? null : s;
}
// use: path + (frag != null ? "#" + safeFragment(frag) : "")

boolean isUriSafeLocalResource(Path p, String symlink) {
  try {
    new URI(p.toUri() + "#" + (symlink == null ? "" : symlink));
    return true;
  } catch (URISyntaxException e) { return false; }
}

When it happens

Trigger: Submitting a job where a -libjars/-archives/-files entry has a symlink name (the part after '#') containing spaces or '#', or where the resolved resource path on the default filesystem contains characters URI cannot represent unencoded — e.g. hadoop jar 'my app.jar#my lib'.

Common situations: Filenames with spaces in shared dirs referenced via -libjars or the distributed cache fragments; User-generated symlink names from templating (e.g. '#${name} ' with trailing whitespace); Paths returned by resolvePath containing reserved characters after unusual FS layouts

Related errors

• Resource name must be relative
• Invalid path URI: {} - cannot contain both a URI fragment an
• Invalid path URI: {} - cannot contain both a URI fragment an
• Error parsing local resource path. Path was not able to be c
• Unrecognized task type: {}

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/6cb23187f1837126. Report an issue: GitHub.