apache/hadoop · error · UnsupportedOperationException

getSnapshotDiff is not supported for HttpFs on {0}. Please c

Error message

getSnapshotDiff is not supported for HttpFs on {0}. Please check your fs.defaultFS configuration

What it means

FSOperations.FSGetSnapshotDiff.execute() computes a snapshot diff report only through DistributedFileSystem.getSnapshotDiffReport(); the generic FileSystem interface has no diff API. When HttpFS's backing filesystem is not HDFS it throws UnsupportedOperationException (HTTP 400 to REST callers), telling you the server-side fs.defaultFS does not point at HDFS.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/fs/http/server/FSOperations.java:1893

      this.path = new Path(path);
      this.oldSnapshotName = oldSnapshotName;
      this.snapshotName = snapshotName;
    }

    /**
     * Executes the filesystem operation.
     * @param fs filesystem instance to use.
     * @return A serialized JSON string of snapshot diffs.
     * @throws IOException thrown if an IO error occurred.
     */
    @Override
    public String execute(FileSystem fs) throws IOException {
      SnapshotDiffReport sdr = null;
      if (fs instanceof DistributedFileSystem) {
        DistributedFileSystem dfs = (DistributedFileSystem) fs;
        sdr = dfs.getSnapshotDiffReport(path, oldSnapshotName, snapshotName);
      } else {
        throw new UnsupportedOperationException("getSnapshotDiff is not "
            + "supported for HttpFs on " + fs.getClass()
            + ". Please check your fs.defaultFS configuration");
      }
      if (sdr != null) {
        return JsonUtil.toJsonString(sdr);
      } else {
        return "";
      }
    }
  }

  /**
   *  Executor that performs a getSnapshotDiffListing operation.
   */
  @InterfaceAudience.Private
  public static class FSGetSnapshotDiffListing implements
      FileSystemAccess.FileSystemExecutor<String> {

View on GitHub (pinned to 2add963021)

Solutions

  1. Set the HttpFS server's fs.defaultFS to the HDFS nameservice and restart
  2. Validate the backing store before enabling snapshot-diff features in tools (e.g., HDFS SnapshotDiff viewers) that default to HttpFS URLs
  3. Bypass HttpFS and use WebHDFS against the NameNode for snapshot diff queries
  4. Confirm both snapshots exist on an HDFS-backed deployment; on non-HDFS backing the op is unavailable regardless of arguments

Example fix

<!-- before: httpfs-site.xml -->
<property><name>fs.defaultFS</name><value>file:///</value></property>
<!-- after -->
<property><name>fs.defaultFS</name><value>hdfs://ns1</value></property>
<!-- restart HttpFS, then GETSNAPSHOTDIFF returns JSON diff report -->
Defensive patterns

Strategy: try-catch

Type guard

static DistributedFileSystem asDfs(FileSystem fs) {
  return fs instanceof DistributedFileSystem ? (DistributedFileSystem) fs : null;
}

Try / catch

try {
  SnapshotDiffReport r = fs.getSnapshotDiffReport(path, oldSnap, newSnap);
} catch (IOException e) {
  if (e.getMessage() != null && e.getMessage().contains("not supported for HttpFs")) {
    // non-HDFS gateway: fetch diffs via WebHDFS on the NameNode instead
    throw new UnsupportedFeatureException("snapshot diff requires HDFS-backed HttpFS", e);
  }
  throw e;
}

Prevention

When it happens

Trigger: Client sends op=GETSNAPSHOTDIFF (oldsnapshotname/snapshotname params) via webhdfs:// while the HttpFS server's filesystem is LocalFileSystem, an object-store FileSystem, or ViewFs.

Common situations: HttpFS proxied to non-HDFS storage in a hybrid lakehouse; missing fs.defaultFS in httpfs config so defaults leak in; snapshot-diff tooling pointed at an HttpFS URL whose server was never HDFS-backed.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/ca3189c0b35c57ca. Report an issue: GitHub.