apache/hadoop · error · PathIOException

Symlinks unsupported

Error message

Symlinks unsupported

What it means

SetReplication.processPath (SetReplication.java:84) throws PathIOException(item, 'Symlinks unsupported') when the enumerated path's FileStatus.isSymlink() is true. Because -setrep runs with setRecursive(true), every child in the tree is checked, and any symlink (file or its target chain) aborts that item. Replication is a property of the target's blocks, so setting it 'through' a link is ambiguous and refused.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/shell/SetReplication.java:84

      displayWarning("Illegal replication, a positive integer expected");
      throw nfe;
    }
    if (newRep < 1) {
      throw new IllegalArgumentException("replication must be >= 1");
    }
  }

  @Override
  protected void processArguments(LinkedList<PathData> args)
  throws IOException {
    super.processArguments(args);
    if (waitOpt) waitForReplication();
  }

  @Override
  protected void processPath(PathData item) throws IOException {
    if (item.stat.isSymlink()) {
      throw new PathIOException(item.toString(), "Symlinks unsupported");
    }
    
    if (item.stat.isFile()) {
      // Do the checking if the file is erasure coded since
      // replication factor for an EC file is meaningless.
      if (!item.stat.isErasureCoded()) {
        if (!item.fs.setReplication(item.path, newRep)) {
          throw new IOException("Could not set replication for: " + item);
        }
        out.println("Replication " + newRep + " set: " + item);
        if (waitOpt) {
          waitList.add(item);
        }
      } else {
        out.println("Did not set replication for: " + item
            + ", because it's an erasure coded file.");
      }
    }

View on GitHub (pinned to 2add963021)

Solutions

  1. Run -setrep on the symlink target path(s) directly instead of through the link
  2. Exclude or remove the symlinks from the tree ('hadoop fs -ls' and filter, or hdfs dfsadmin-level tooling) before the recursive setrep
  3. In code, pre-check FileStatus.isSymlink() and resolve via getSymlinkLink()/Path(target) before calling setReplication

Example fix

# before
hadoop fs -setrep -w 3 /data/manifest   # contains a symlink
# setrep: Symlinks unsupported

# after
hadoop fs -ls /data/manifest            # identify the link
hadoop fs -setrep -w 3 /data/manifest/real-file-1 /data/manifest/real-file-2
Defensive patterns

Strategy: type-guard

Validate before calling

for (FileStatus st : fs.listStatus(dir)) {
  if (st.isSymlink()) continue; // or resolve: new Path(st.getSymlink())
  fs.setReplication(st.getPath(), rep);
}

Type guard

boolean setrepSafe(FileStatus st) {
  return !st.isSymlink() && st.isFile() && !st.isErasureCoded();
}

Try / catch

catch (PathIOException e) when 'Symlinks unsupported' -> resolve the link target via FileStatus.getSymlink() and setReplication on the target, or skip

Prevention

When it happens

Trigger: 'hadoop fs -setrep 3 /dir' where /dir or any file under it is a symlink (created via FileSystem.createSymlink / -put with symlinks preserved); running setrep on an HDFS tree containing symlinked entries.

Common situations: Ingestion trees that materialize symlinks (e.g., manifest-style datasets, Hive-style symlink tables); migrating data with symlinked convenience names; EC/replication sweeps over user directories where users added symlinks.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/a155735086d297fc. Report an issue: GitHub.