apache/hadoop · error · IOException

Unable to support FSFileBlockLocationsLegacy because the fil

Error message

Unable to support FSFileBlockLocationsLegacy because the file system is not DistributedFileSystem.

What it means

FSOperations.FSFileBlockLocationsLegacy.execute() serves the legacy block-locations request by calling DistributedFileSystem.getLocatedBlocks() and JSON-serializing LocatedBlocks; the legacy wire shape cannot be produced from the generic FileSystem API. When the backing filesystem is not DistributedFileSystem it throws plain IOException (not UnsupportedOperationException), which HttpExceptionUtils maps to HTTP 500 rather than 400.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/fs/http/server/FSOperations.java:2284

     * @param path the path to retrieve the location
     * @param offsetValue offset into the given file
     * @param lengthValue length for which to get locations for
     */
    public FSFileBlockLocationsLegacy(String path, long offsetValue, long lengthValue) {
      this.path = new Path(path);
      this.offsetValue = offsetValue;
      this.lengthValue = lengthValue;
    }

    @Override
    public Map execute(FileSystem fs) throws IOException {
      if (fs instanceof DistributedFileSystem) {
        DistributedFileSystem dfs = (DistributedFileSystem)fs;
        LocatedBlocks locations = dfs.getLocatedBlocks(
            this.path, this.offsetValue, this.lengthValue);
        return JsonUtil.toJsonMap(locations);
      }
      throw new IOException("Unable to support FSFileBlockLocationsLegacy " +
          "because the file system is not DistributedFileSystem.");
    }
  }

  /**
   * Executor that performs a linkFile-status FileSystemAccess files
   * system operation.
   */
  @InterfaceAudience.Private
  @SuppressWarnings("rawtypes")
  public static class FSFileLinkStatus
      implements FileSystemAccess.FileSystemExecutor<Map> {
    final private Path path;

    /**
     * Creates a linkFile-status executor.
     *
     * @param path the path to retrieve the status.

View on GitHub (pinned to 2add963021)

Solutions

  1. Configure the HttpFS server's fs.defaultFS to the HDFS nameservice and restart it
  2. Upgrade the client to a Hadoop version that uses the non-legacy block-locations call, which has broader filesystem support
  3. Compute splits against WebHDFS on the NameNode directly for HDFS data
  4. Distinguish this 500-capability error from real server faults by the message text before debugging deeper

Example fix

<!-- before: httpfs-site.xml -->
<property><name>fs.defaultFS</name><value>file:///</value></property>
<!-- after -->
<property><name>fs.defaultFS</name><value>hdfs://ns1</value></property>
<!-- restart HttpFS; legacy GETFILEBLOCKLOCATIONS then returns LocatedBlocks JSON -->
Defensive patterns

Strategy: try-catch

Type guard

static DistributedFileSystem asDfs(FileSystem fs) {
  return fs instanceof DistributedFileSystem ? (DistributedFileSystem) fs : null;
}

Try / catch

try {
  BlockLocation[] locs = fs.getFileBlockLocations(path, 0, len); // legacy path via HttpFS
} catch (IOException e) {
  if (e.getMessage() != null && e.getMessage().contains("not DistributedFileSystem")) {
    // legacy executor unavailable: compute splits against WebHDFS on the NameNode
    throw new UnsupportedFeatureException("legacy block locations need HDFS-backed HttpFS", e);
  }
  throw e;
}

Prevention

When it happens

Trigger: An older WebHDFS client requests op=GETFILEBLOCKLOCATIONS through an HttpFS server whose fs.defaultFS is file:/// or an object store, hitting the legacy executor instead of the modern block-locations path.

Common situations: Legacy MapReduce/scheduling tools computing input splits via HttpFS; mixed clusters where some gateways are HDFS-backed and others are not; upgrades that left old client jars asking for the legacy format.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/b18e80ccf7a9e578. Report an issue: GitHub.