apache/hadoop · error · IllegalArgumentException

Resource name must be relative

Error message

Resource name must be relative

What it means

In LocalResourceBuilder.createLocalResources, when a cache URI has no wildcard the URI fragment (the part after '#') is used as the symlink name for the localized file. Because the fragment is attacker/untrusted-shaped input, the builder validates it: if new Path(linkName).isAbsolute() it throws IllegalArgumentException('Resource name must be relative'). Absolute symlink targets inside the sandboxed working directory are meaningless and unsafe, so they are rejected up front.

Source

Thrown at hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/util/LocalResourceBuilder.java:128

        if (p.getName().equals(DistributedCache.WILDCARD)) {
          p = p.getParent();
          linkName = p.getName() + Path.SEPARATOR + DistributedCache.WILDCARD;
        }

        p = remoteFS.resolvePath(p.makeQualified(remoteFS.getUri(),
            remoteFS.getWorkingDirectory()));

        // If there's no wildcard, try using the fragment for the link
        if (linkName == null) {
          linkName = u.getFragment();

          // Because we don't know what's in the fragment, we have to handle
          // it with care.
          if (linkName != null) {
            Path linkPath = new Path(linkName);

            if (linkPath.isAbsolute()) {
              throw new IllegalArgumentException("Resource name must be "
                  + "relative");
            }

            linkName = linkPath.toUri().getPath();
          }
        } else if (u.getFragment() != null) {
          throw new IllegalArgumentException("Invalid path URI: " + p +
              " - cannot contain both a URI fragment and a wildcard");
        }

        // If there's no wildcard or fragment, just link to the file name
        if (linkName == null) {
          linkName = p.getName();
        }

        LocalResource orig = localResources.get(linkName);
        if(orig != null && !orig.getResource().equals(URL.fromURI(p.toUri()))) {
          LOG.warn(getResourceDescription(orig.getType()) + orig.getResource()

View on GitHub (pinned to 2add963021)

Solutions

1. Use a plain relative name as the fragment: '#lib.so' — the file localizes into the task working dir under that name
2. Drop the fragment entirely to let the link name default to the file name (p.getName())
3. If a specific absolute path is required, symlink it from the task's working directory after localization

Example fix

// before
job.addCacheFile(new URI("hdfs://nn/native/libhadoop.so#/usr/lib/libhadoop.so")); // throws

// after
job.addCacheFile(new URI("hdfs://nn/native/libhadoop.so#libhadoop.so"));
// then reference it as ./libhadoop.so from the task working directory

static String relativeFragmentOrNull(URI u) {
  String f = u.getFragment();
  if (f == null) return null;
  Path link = new Path(f);
  if (link.isAbsolute()) {
    throw new IllegalArgumentException("Fragment must be relative: " + u);
  }
  return f;
}
// call before job.addCacheFile(u)

try {
  job.addCacheFile(new URI(rawUri));
} catch (IllegalArgumentException e) {
  throw new IllegalArgumentException("Bad cache URI '" + rawUri
      + "': use a relative #fragment, e.g. file.jar#file.jar", e);
}

When it happens

Trigger: A distributed-cache entry whose fragment begins with '/' (or is otherwise an absolute path), e.g. 'hdfs://nn/lib/lib.so#/tmp/lib.so' passed to job.addCacheFile or mapreduce.job.cache.files.

Common situations: Users trying to control where the file appears by putting an absolute path after '#'; porting shell scripts that reference fixed paths; fragments accidentally containing a leading slash from string concatenation.

Related errors

• Invalid path URI: {} - cannot contain both a URI fragment an
• Invalid path URI: {} - cannot contain both a URI fragment an
• Failed to create a URI (URISyntaxException) for the remote p
• Failed to create a URI (URISyntaxException) for the remote p
• Failed to create a URI (URISyntaxException) for the remote p

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/d79179d59ad5749d. Report an issue: GitHub.