apache/hadoop · error · UnsupportedOperationException

This API:{} is specific to DFS. Can't run on other fs:{}

Error message

This API:{} is specific to DFS. Can't run on other fs:{}

What it means

checkDFS(fs, methodName) throws UnsupportedOperationException when the filesystem resolved for a path is not a DistributedFileSystem. DFS-only APIs exposed on ViewDistributedFileSystem route through this guard, and mounts backed by non-HDFS filesystems (object stores, LocalFS) cannot serve them.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/ViewDistributedFileSystem.java:399

      final EnumSet<CreateFlag> cflags, final int bufferSize,
      final short replication, final long blockSize,
      final Progressable progress, final Options.ChecksumOpt checksumOpt)
      throws IOException {
    if (this.vfs == null) {
      return super
          .create(f, permission, cflags, bufferSize, replication, blockSize,
              progress, checksumOpt);
    }
    return vfs.create(f, permission, cflags, bufferSize, replication, blockSize,
        progress, checksumOpt);
  }

  void checkDFS(FileSystem fs, String methodName) {
    if (!(fs instanceof DistributedFileSystem)) {
      String msg = new StringBuilder("This API:").append(methodName)
          .append(" is specific to DFS. Can't run on other fs:")
          .append(fs.getUri()).toString();
      throw new UnsupportedOperationException(msg);
    }
  }

  void checkDefaultDFS(FileSystem fs, String methodName) {
    if (fs == null) {
      String msg = new StringBuilder("This API:").append(methodName).append(
          " cannot be supported without default cluster(that is linkFallBack).")
          .toString();
      throw new UnsupportedOperationException(msg);
    }
  }

  @Override
  // DFS specific API
  protected HdfsDataOutputStream primitiveCreate(Path f,
      FsPermission absolutePermission, EnumSet<CreateFlag> flag, int bufferSize,
      short replication, long blockSize, Progressable progress,
      Options.ChecksumOpt checksumOpt) throws IOException {

View on GitHub (pinned to 2add963021)

Solutions

  1. Call the API on the concrete target filesystem of that mount instead of through the view
  2. Restrict DFS-only operations to paths mounted on real HDFS clusters
  3. Catch UnsupportedOperationException and degrade gracefully for non-DFS mounts

Example fix

// before
viewFs.setStoragePolicy(path, "COLD");

// after: resolve the mount target and call the DFS API on it
ViewFileSystemOverloadScheme.MountPathInfo<FileSystem> info =
    ((ViewDistributedFileSystem) viewFs).getMountPathInfo(path, conf);
if (info.getTargetFs() instanceof DistributedFileSystem) {
  ((DistributedFileSystem) info.getTargetFs())
      .setStoragePolicy(info.getPathOnTarget(), "COLD");
}
Defensive patterns

Strategy: validation

Validate before calling

ViewFileSystemOverloadScheme.MountPathInfo<FileSystem> info =
    ((ViewDistributedFileSystem) viewFs).getMountPathInfo(path, conf);
if (!(info.getTargetFs() instanceof DistributedFileSystem)) {
  throw new IllegalArgumentException(
      "path " + path + " is not on an HDFS mount");
}
// safe to call DFS-only API on info.getTargetFs()

Type guard

static boolean isOnDfsMount(FileSystem viewFs, Path p, Configuration conf) {
  if (!(viewFs instanceof ViewDistributedFileSystem)) {
    return viewFs instanceof DistributedFileSystem;
  }
  return ((ViewDistributedFileSystem) viewFs)
      .getMountPathInfo(p, conf).getTargetFs()
      instanceof DistributedFileSystem;
}

Try / catch

try {
  dfsOnlyApi(viewFs, path);
} catch (UnsupportedOperationException e) {
  if (e.getMessage() != null && e.getMessage().contains("specific to DFS")) {
    // path resolved to a non-HDFS mount; route or skip
  } else {
    throw e;
  }
}

Prevention

When it happens

Trigger: Invoking a DFS-only API (storage policy, quota, snapshot operations guarded by checkDFS) through ViewHDFS where the path resolves to a mount whose target filesystem is not DistributedFileSystem.

Common situations: Mount tables mixing HDFS and object-store mounts; legacy admin scripts assuming every mount is HDFS; running DFS-specific commands on viewfs paths that land on non-HDFS targets.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/5f25b3845f7bc4c1. Report an issue: GitHub.