apache/hadoop · error · UnsupportedOperationException

setErasureCodingPolicy is not supported for HttpFs on {0}. P

Error message

setErasureCodingPolicy is not supported for HttpFs on {0}. Please check your fs.defaultFS configuration

What it means

FSOperations.FSSetErasureCodingPolicy.execute() applies SETERASURECODINGPOLICY only via DistributedFileSystem.setErasureCodingPolicy(); erasure coding is an HDFS-native feature with no generic FileSystem API. Any non-HDFS backing filesystem yields UnsupportedOperationException (HTTP 400 with this text), steering you to the server's fs.defaultFS.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/fs/http/server/FSOperations.java:2128

  @InterfaceAudience.Private
  public static class FSSetErasureCodingPolicy
      implements FileSystemAccess.FileSystemExecutor<Void> {

    private Path path;
    private String policyName;

    public FSSetErasureCodingPolicy(String path, String policyName) {
      this.path = new Path(path);
      this.policyName = policyName;
    }

    @Override
    public Void execute(FileSystem fs) throws IOException {
      if (fs instanceof DistributedFileSystem) {
        DistributedFileSystem dfs = (DistributedFileSystem) fs;
        dfs.setErasureCodingPolicy(path, policyName);
      } else {
        throw new UnsupportedOperationException("setErasureCodingPolicy is "
            + "not supported for HttpFs on " + fs.getClass()
            + ". Please check your fs.defaultFS configuration");
      }
      return null;
    }
  }

  /**
   * Executor that performs a getErasureCodingPolicy operation.
   */
  @InterfaceAudience.Private
  public static class FSGetErasureCodingPolicy
      implements FileSystemAccess.FileSystemExecutor<String> {

    private Path path;

    public FSGetErasureCodingPolicy(String path) {
      this.path = new Path(path);

View on GitHub (pinned to 2add963021)

Solutions

  1. Point the HttpFS server's fs.defaultFS at HDFS and restart it
  2. Apply EC policies via WebHDFS/hdfs CLI directly on the HDFS cluster
  3. Confirm the directory is a writable HDFS dir (EC policies require empty dirs and EC-enabled zones)
  4. If the target store is not HDFS, drop the EC step: object stores manage redundancy themselves

Example fix

<!-- before: httpfs-site.xml -->
<property><name>fs.defaultFS</name><value>file:///</value></property>
<!-- after -->
<property><name>fs.defaultFS</name><value>hdfs://ns1</value></property>
<!-- restart HttpFS; SETERASURECODINGPOLICY then reaches DFS -->
Defensive patterns

Strategy: try-catch

Type guard

static DistributedFileSystem asDfs(FileSystem fs) {
  return fs instanceof DistributedFileSystem ? (DistributedFileSystem) fs : null;
}

Try / catch

try {
  fs.setErasureCodingPolicy(path, "RS-6-3-1024k");
} catch (IOException e) {
  if (e.getMessage() != null && e.getMessage().contains("not supported for HttpFs")) {
    throw new UnsupportedFeatureException("EC requires HDFS-backed HttpFS; fix fs.defaultFS or use WebHDFS", e);
  }
  throw e;
}

Prevention

When it happens

Trigger: Client sends op=SETERASURECODINGPOLICY&ecpolicy=RS-6-3-1024k via webhdfs:// while HttpFS is backed by local FS, s3a, or another non-HDFS store.

Common situations: EC-enablement scripts run against an HttpFS URL on a gateway that fronts object storage; server config drift after moving workloads off HDFS; HttpFS test instance without HDFS.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/945a5c6d42714d57. Report an issue: GitHub.