apache/hadoop · error · UnsupportedOperationException

setErasureCodingPolicy is not supported for HttpFs on

Error message

setErasureCodingPolicy is not supported for HttpFs on {0}. Please check your fs.defaultFS configuration

What it means

FSOperations.FSSetErasureCodingPolicy.execute() applies SETERASURECODINGPOLICY only via DistributedFileSystem.setErasureCodingPolicy(); erasure coding is an HDFS-native feature with no generic FileSystem API. Any non-HDFS backing filesystem yields UnsupportedOperationException (HTTP 400 with this text), steering you to the server's fs.defaultFS.

Solutions

  1. Point the HttpFS server's fs.defaultFS at HDFS and restart it
  2. Apply EC policies via WebHDFS/hdfs CLI directly on the HDFS cluster
  3. Confirm the directory is a writable HDFS dir (EC policies require empty dirs and EC-enabled zones)
  4. If the target store is not HDFS, drop the EC step: object stores manage redundancy themselves

Example fix

<!-- before: httpfs-site.xml -->
<property><name>fs.defaultFS</name><value>file:///</value></property>
<!-- after -->
<property><name>fs.defaultFS</name><value>hdfs://ns1</value></property>
<!-- restart HttpFS; SETERASURECODINGPOLICY then reaches DFS -->
Defensive patterns

Strategy: try-catch

Type guard

static DistributedFileSystem asDfs(FileSystem fs) {
  return fs instanceof DistributedFileSystem ? (DistributedFileSystem) fs : null;
}

Try / catch

try {
  fs.setErasureCodingPolicy(path, "RS-6-3-1024k");
} catch (IOException e) {
  if (e.getMessage() != null && e.getMessage().contains("not supported for HttpFs")) {
    throw new UnsupportedFeatureException("EC requires HDFS-backed HttpFS; fix fs.defaultFS or use WebHDFS", e);
  }
  throw e;
}

Prevention

When it happens

Trigger: Client sends op=SETERASURECODINGPOLICY&ecpolicy=RS-6-3-1024k via webhdfs:// while HttpFS is backed by local FS, s3a, or another non-HDFS store.

Common situations: EC-enablement scripts run against an HttpFS URL on a gateway that fronts object storage; server config drift after moving workloads off HDFS; HttpFS test instance without HDFS.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/945a5c6d42714d57. Report an issue: GitHub.

Appendix: source

Thrown at hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/fs/http/server/FSOperations.java:2128

  @InterfaceAudience.Private
  public static class FSSetErasureCodingPolicy
      implements FileSystemAccess.FileSystemExecutor<Void> {

    private Path path;
    private String policyName;

    public FSSetErasureCodingPolicy(String path, String policyName) {
      this.path = new Path(path);
      this.policyName = policyName;
    }

    @Override
    public Void execute(FileSystem fs) throws IOException {
      if (fs instanceof DistributedFileSystem) {
        DistributedFileSystem dfs = (DistributedFileSystem) fs;
        dfs.setErasureCodingPolicy(path, policyName);
      } else {
        throw new UnsupportedOperationException("setErasureCodingPolicy is "
            + "not supported for HttpFs on " + fs.getClass()
            + ". Please check your fs.defaultFS configuration");
      }
      return null;
    }
  }

  /**
   * Executor that performs a getErasureCodingPolicy operation.
   */
  @InterfaceAudience.Private
  public static class FSGetErasureCodingPolicy
      implements FileSystemAccess.FileSystemExecutor<String> {

    private Path path;

    public FSGetErasureCodingPolicy(String path) {
      this.path = new Path(path);

View on GitHub (pinned to 2add963021)