apache/hadoop · error · UnsupportedFileSystemException

getFileChecksum(Path, long) is not supported by

Error message

getFileChecksum(Path, long) is not supported by 

What it means

getFileChecksum(Path, long) with a length argument uses HDFS-specific combined-mode checksums (dfs.getFileChecksumWithCombineMode). When a symlink resolves onto a filesystem that is not a DistributedFileSystem, the generic FileSystem contract cannot express this operation, so the resolver throws UnsupportedFileSystemException naming the target filesystem class.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java:2085

  @Override
  public FileChecksum getFileChecksum(Path f, final long length)
      throws IOException {
    statistics.incrementReadOps(1);
    storageStatistics.incrementOpCounter(OpType.GET_FILE_CHECKSUM);
    Path absF = fixRelativePart(f);
    return new FileSystemLinkResolver<FileChecksum>() {
      @Override
      public FileChecksum doCall(final Path p) throws IOException {
        return dfs.getFileChecksumWithCombineMode(getPathName(p), length);
      }

      @Override
      public FileChecksum next(final FileSystem fs, final Path p)
          throws IOException {
        if (fs instanceof DistributedFileSystem) {
          return fs.getFileChecksum(p, length);
        } else {
          throw new UnsupportedFileSystemException(
              "getFileChecksum(Path, long) is not supported by "
                  + fs.getClass().getSimpleName());
        }
      }
    }.resolve(this, absF);
  }

  @Override
  public void setPermission(Path p, final FsPermission permission
  ) throws IOException {
    statistics.incrementWriteOps(1);
    storageStatistics.incrementOpCounter(OpType.SET_PERMISSION);
    Path absF = fixRelativePart(p);
    new FileSystemLinkResolver<Void>() {
      @Override
      public Void doCall(final Path p) throws IOException {
        dfs.setPermission(getPathName(p), permission);
        return null;

View on GitHub (pinned to 2add963021)

Solutions

1. Resolve the link target yourself and only call getFileChecksum(p, length) on the DistributedFileSystem that owns the target.
2. Use the single-argument getFileChecksum(p), which is part of the generic FileSystem API and works cross-filesystem via fallback.
3. Restructure so checksum verification never traverses cross-filesystem symlinks.
4. Skip/flag foreign-scheme paths in verification jobs instead of failing the whole run.

Example fix

// before
FileChecksum cs = hdfs.getFileChecksum(linkPath, length); // UnsupportedFileSystemException

// after
Path target = hdfs.getLinkTarget(linkPath);
FileSystem targetFs = FileSystem.get(target.toUri(), conf);
FileChecksum cs = targetFs instanceof DistributedFileSystem
    ? targetFs.getFileChecksum(target, length)
    : targetFs.getFileChecksum(target);

Path target = fs.getLinkTarget(p);
FileSystem tfs = FileSystem.get(target.toUri(), conf);
if (!(tfs instanceof DistributedFileSystem)) {
  // plan to use single-arg getFileChecksum on tfs instead
}

try {
  cs = fs.getFileChecksum(p, length);
} catch (UnsupportedFileSystemException e) {
  cs = fs.getFileChecksum(p); // generic API works cross-filesystem
}

When it happens

Trigger: Calling fs.getFileChecksum(p, length) where p is an HDFS symlink whose target resolves onto a non-HDFS filesystem (file://, s3a://, etc.), so next(fs, p) receives a foreign FileSystem instance.

Common situations: Data-integrity tools computing partial checksums over hybrid lakes with cross-scheme symlinks; checksum verification code that worked on plain HDFS paths after symlinks were introduced by a re-layout; mount-table (viewfs) setups routing some paths to object stores.

Related errors

• Link resolution does not work with multiple file systems for
• FileSystem does not support non-recursivemkdir
• Cannot perform snapshot operations on a symlink to a non-Dis
• Symlinks not supported
• Filesystem does not support symlinks!

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/c411c75a393d8dc3. Report an issue: GitHub.