apache/hadoop · error · IllegalArgumentException

Resource name must be relative

Error message

Resource name must be relative

What it means

In LocalResourceBuilder.createLocalResources, when a cache URI has no wildcard the URI fragment (the part after '#') is used as the symlink name for the localized file. Because the fragment is attacker/untrusted-shaped input, the builder validates it: if new Path(linkName).isAbsolute() it throws IllegalArgumentException('Resource name must be relative'). Absolute symlink targets inside the sandboxed working directory are meaningless and unsafe, so they are rejected up front.

Solutions

  1. Use a plain relative name as the fragment: '#lib.so' — the file localizes into the task working dir under that name
  2. Drop the fragment entirely to let the link name default to the file name (p.getName())
  3. If a specific absolute path is required, symlink it from the task's working directory after localization

Example fix

// before
job.addCacheFile(new URI("hdfs://nn/native/libhadoop.so#/usr/lib/libhadoop.so")); // throws

// after
job.addCacheFile(new URI("hdfs://nn/native/libhadoop.so#libhadoop.so"));
// then reference it as ./libhadoop.so from the task working directory
Defensive patterns

Strategy: validation

Validate before calling

static String relativeFragmentOrNull(URI u) {
  String f = u.getFragment();
  if (f == null) return null;
  Path link = new Path(f);
  if (link.isAbsolute()) {
    throw new IllegalArgumentException("Fragment must be relative: " + u);
  }
  return f;
}
// call before job.addCacheFile(u)

Try / catch

try {
  job.addCacheFile(new URI(rawUri));
} catch (IllegalArgumentException e) {
  throw new IllegalArgumentException("Bad cache URI '" + rawUri
      + "': use a relative #fragment, e.g. file.jar#file.jar", e);
}

Prevention

When it happens

Trigger: A distributed-cache entry whose fragment begins with '/' (or is otherwise an absolute path), e.g. 'hdfs://nn/lib/lib.so#/tmp/lib.so' passed to job.addCacheFile or mapreduce.job.cache.files.

Common situations: Users trying to control where the file appears by putting an absolute path after '#'; porting shell scripts that reference fixed paths; fragments accidentally containing a leading slash from string concatenation.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/d79179d59ad5749d. Report an issue: GitHub.

Appendix: source

Thrown at hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/util/LocalResourceBuilder.java:128

        if (p.getName().equals(DistributedCache.WILDCARD)) {
          p = p.getParent();
          linkName = p.getName() + Path.SEPARATOR + DistributedCache.WILDCARD;
        }

        p = remoteFS.resolvePath(p.makeQualified(remoteFS.getUri(),
            remoteFS.getWorkingDirectory()));

        // If there's no wildcard, try using the fragment for the link
        if (linkName == null) {
          linkName = u.getFragment();

          // Because we don't know what's in the fragment, we have to handle
          // it with care.
          if (linkName != null) {
            Path linkPath = new Path(linkName);

            if (linkPath.isAbsolute()) {
              throw new IllegalArgumentException("Resource name must be "
                  + "relative");
            }

            linkName = linkPath.toUri().getPath();
          }
        } else if (u.getFragment() != null) {
          throw new IllegalArgumentException("Invalid path URI: " + p +
              " - cannot contain both a URI fragment and a wildcard");
        }

        // If there's no wildcard or fragment, just link to the file name
        if (linkName == null) {
          linkName = p.getName();
        }

        LocalResource orig = localResources.get(linkName);
        if(orig != null && !orig.getResource().equals(URL.fromURI(p.toUri()))) {
          LOG.warn(getResourceDescription(orig.getType()) + orig.getResource()

View on GitHub (pinned to 2add963021)