apache/hadoop · error · UnsupportedOperationException

getErasureCodingPolicies is not supported for HttpFs on

Error message

getErasureCodingPolicies is not supported for HttpFs on {0}. Please check your fs.defaultFS configuration

What it means

FSOperations.FSGetErasureCodingPolicies.execute() lists all EC policies via DistributedFileSystem.getAllErasureCodingPolicies(); like the other EC executors it refuses non-HDFS backing filesystems with UnsupportedOperationException (HTTP 400 to REST clients). The metrics increment (incrOpsAllECPolicies) happens only after the DFS branch succeeds, so failing calls leave the counter untouched.

Solutions

Set fs.defaultFS=hdfs://<nameservice> in the HttpFS server configuration and restart
Probe capabilities via WebHDFS against the NameNode instead of the gateway
Treat this error as a hard capability signal: cache it and stop issuing EC calls to this endpoint
Verify the running server's effective config from its startup log, not just the on-disk file

Example fix

<!-- before: httpfs-site.xml -->
<property><name>fs.defaultFS</name><value>file:///</value></property>
<!-- after -->
<property><name>fs.defaultFS</name><value>hdfs://ns1</value></property>
<!-- restart HttpFS; GETERASURECODINGPOLICIES then returns the policy list JSON -->

Defensive patterns

Strategy: try-catch

Type guard

static DistributedFileSystem asDfs(FileSystem fs) {
  return fs instanceof DistributedFileSystem ? (DistributedFileSystem) fs : null;
}

Try / catch

try {
  Collection<ErasureCodingPolicyInfo> ps = fs.getAllErasureCodingPolicies();
} catch (IOException e) {
  if (e.getMessage() != null && e.getMessage().contains("not supported for HttpFs")) {
    return Collections.emptyList(); // gateway not HDFS-backed: no EC policies exist
  }
  throw e;
}

Prevention

Use this list call as a capability probe; empty-on-unsupported keeps UIs alive
Cache the capability result per endpoint instead of re-probing every request
Expose EC management only through HDFS-backed endpoints (WebHDFS on the NameNode)

When it happens

Trigger: Client sends op=GETERASURECODINGPOLICIES via webhdfs:// while the HttpFS server's FileSystemAccess layer opened a non-HDFS filesystem from its fs.defaultFS.

Common situations: Cluster-capability probes ('what EC policies exist') routed through an HttpFS gateway not backed by HDFS; management UIs defaulting to HttpFS; server config missing or stale after storage migration.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/dee44a5a0f8f26cb. Report an issue: GitHub.

Appendix: source

Thrown at hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/fs/http/server/FSOperations.java:2365

  /**
   * Executor that performs a FSGetErasureCodingPolicies operation.
   */
  @InterfaceAudience.Private
  public static class FSGetErasureCodingPolicies
      implements FileSystemAccess.FileSystemExecutor<String> {

    public FSGetErasureCodingPolicies() {
    }

    @Override
    public String execute(FileSystem fs) throws IOException {
      Collection<ErasureCodingPolicyInfo> ecPolicyInfos = null;
      if (fs instanceof DistributedFileSystem) {
        DistributedFileSystem dfs = (DistributedFileSystem) fs;
        ecPolicyInfos = dfs.getAllErasureCodingPolicies();
      } else {
        throw new UnsupportedOperationException("getErasureCodingPolicies is " +
            "not supported for HttpFs on " + fs.getClass() +
            ". Please check your fs.defaultFS configuration");
      }
      HttpFSServerWebApp.get().getMetrics().incrOpsAllECPolicies();
      return JsonUtil.toJsonString(ecPolicyInfos.stream().toArray(ErasureCodingPolicyInfo[]::new));
    }
  }

  /**
   * Executor that performs a FSGetErasureCodingCodecs operation.
   */
  @InterfaceAudience.Private
  public static class FSGetErasureCodingCodecs
      implements FileSystemAccess.FileSystemExecutor<Map> {

    public FSGetErasureCodingCodecs() {
    }

View on GitHub (pinned to 2add963021)