What causes "Symlinks not supported" in apache/hadoop?

For local paths, bypass Hadoop and use java.nio.file.Files.createSymbolicLink directly

apache/hadoop · error · UnsupportedOperationException

Symlinks not supported

Error message

Symlinks not supported

What it means

Thrown by RawLocalFileSystem.createSymlink when FileSystem.areSymlinksEnabled() returns false. Symlinks in Hadoop FileSystem are disabled globally by default (symlinksEnabled = false, see HADOOP-10020/HADOOP-10052) because supporting them across all FileSystems broke path resolution; the enable method is @VisibleForTesting. In any production JVM this throws UnsupportedOperationException on the first call.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/RawLocalFileSystem.java:1266

    }
    final Path p = stat.getPath();
    final Optional<Long> mtime = !data.allowChange()
        ? Optional.of(stat.getModificationTime())
        : Optional.empty();
    return new LocalFileSystemPathHandle(p.toString(), mtime);
  }

  @Override
  public boolean supportsSymlinks() {
    return true;
  }

  @SuppressWarnings("deprecation")
  @Override
  public void createSymlink(Path target, Path link, boolean createParent)
      throws IOException {
    if (!FileSystem.areSymlinksEnabled()) {
      throw new UnsupportedOperationException("Symlinks not supported");
    }
    final String targetScheme = target.toUri().getScheme();
    if (targetScheme != null && !"file".equals(targetScheme)) {
      throw new IOException("Unable to create symlink to non-local file "+
                            "system: "+target.toString());
    }
    if (createParent) {
      mkdirs(link.getParent());
    }

    // NB: Use createSymbolicLink in java.nio.file.Path once available
    int result = FileUtil.symLink(target.toString(),
        makeAbsolute(link).toString());
    if (result != 0) {
      throw new IOException("Error " + result + " creating symlink " +
          link + " to " + target);
    }
  }

View on GitHub (pinned to 2add963021)

Solutions

For local paths, bypass Hadoop and use java.nio.file.Files.createSymbolicLink directly
In tests, call FileSystem.enableSymlinks() once in @BeforeClass/@BeforeAll
Replace symlink usage with hard links (FileUtil.hardLink), copies, or manifest files referencing alternate locations

Example fix

// before
rawFs.createSymlink(target, link, true); // UnsupportedOperationException by default

// after
java.nio.file.Files.createSymbolicLink(
    new java.io.File(link.toUri().getPath()).toPath(),
    new java.io.File(target.toUri().getPath()).toPath());

Defensive patterns

Strategy: validation

Validate before calling

if (!FileSystem.areSymlinksEnabled()) {
  // use java.nio for local paths; do not call fs.createSymlink
  java.nio.file.Files.createSymbolicLink(
      java.nio.file.Paths.get(link.toUri().getPath()),
      java.nio.file.Paths.get(target.toUri().getPath()));
} else {
  fs.createSymlink(target, link, true);
}

Try / catch

try {
  fs.createSymlink(target, link, true);
} catch (UnsupportedOperationException e) {
  // symlinks globally disabled (HADOOP-10020); fall back to NIO for local paths
  java.nio.file.Files.createSymbolicLink(
      java.nio.file.Paths.get(link.toUri().getPath()),
      java.nio.file.Paths.get(target.toUri().getPath()));
}

Prevention

Do not use Hadoop symlink APIs in production code; they are disabled by default
In tests, call FileSystem.enableSymlinks() in the setup phase, and never let that setup leak into production paths

When it happens

Trigger: Calling FileSystem.createSymlink(target, link, createParent) or FileContext.createSymlink on RawLocalFileSystem without a prior test-only FileSystem.enableSymlinks() call in the same JVM.

Common situations: Running old MapReduce/user code written before symlinks were disabled (Hadoop 2.x era); unit tests that forgot the FileSystem.enableSymlinks() setup; code copied from test classes into production jobs.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/bc5d83ef73335c96. Report an issue: GitHub.