apache/hadoop · error · UnsupportedOperationException

unsetErasureCodingPolicy is not supported for HttpFs on {0}.

Error message

unsetErasureCodingPolicy is not supported for HttpFs on {0}. Please check your fs.defaultFS configuration

What it means

FSOperations.FSUnSetErasureCodingPolicy.execute() handles UNSETERASURECODINGPOLICY only on DistributedFileSystem, since EC policy removal is an HDFS-specific operation. With a non-HDFS backing filesystem it throws UnsupportedOperationException, which HttpExceptionUtils converts to HTTP 400 for REST callers, and the message points at the server-side fs.defaultFS.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/fs/http/server/FSOperations.java:2183

   * Executor that performs a unsetErasureCodingPolicy operation.
   */
  @InterfaceAudience.Private
  public static class FSUnSetErasureCodingPolicy
      implements FileSystemAccess.FileSystemExecutor<Void> {

    private Path path;

    public FSUnSetErasureCodingPolicy(String path) {
      this.path = new Path(path);
    }

    @Override
    public Void execute(FileSystem fs) throws IOException {
      if (fs instanceof DistributedFileSystem) {
        DistributedFileSystem dfs = (DistributedFileSystem) fs;
        dfs.unsetErasureCodingPolicy(path);
      } else {
        throw new UnsupportedOperationException("unsetErasureCodingPolicy is "
            + "not supported for HttpFs on " + fs.getClass()
            + ". Please check your fs.defaultFS configuration");
      }
      return null;
    }
  }

  /**
   * Executor that performs a satisfyStoragePolicy operation.
   */
  @InterfaceAudience.Private
  public static class FSSatisyStoragePolicy
      implements FileSystemAccess.FileSystemExecutor<Void> {

    private Path path;

    public FSSatisyStoragePolicy(String path) {
      this.path = new Path(path);

View on GitHub (pinned to 2add963021)

Solutions

Configure fs.defaultFS=hdfs://<nameservice> on the HttpFS server and restart
Run the unset via hdfs CLI or WebHDFS directly on the HDFS cluster
Make the client script tolerant of mixed backends: detect this error and skip instead of aborting the run
Audit httpfs-site.xml for the effective default filesystem rather than trusting host core-site.xml

Example fix

<!-- before: httpfs-site.xml -->
<property><name>fs.defaultFS</name><value>file:///</value></property>
<!-- after -->
<property><name>fs.defaultFS</name><value>hdfs://ns1</value></property>
<!-- restart HttpFS; UNSETERASURECODINGPOLICY then works on HDFS paths -->

Defensive patterns

Strategy: try-catch

Type guard

static DistributedFileSystem asDfs(FileSystem fs) {
  return fs instanceof DistributedFileSystem ? (DistributedFileSystem) fs : null;
}

Try / catch

try {
  fs.unsetErasureCodingPolicy(path);
} catch (IOException e) {
  if (e.getMessage() != null && e.getMessage().contains("not supported for HttpFs")) {
    LOG.info("Skipping unsetErasureCodingPolicy({}): gateway not HDFS-backed", path);
    return;
  }
  throw e;
}

Prevention

Make cleanup scripts idempotent and capability-aware: skip on 'not supported for HttpFs'
Run EC lifecycle operations against the HDFS cluster directly
Keep a registry of which gateways are HDFS-backed for automation

When it happens

Trigger: Client sends op=UNSETERASURECODINGPOLICY via webhdfs:// while the HttpFS gateway's filesystem is LocalFileSystem or an object store.

Common situations: Cleanup scripts reverting EC policies through the wrong gateway URL; HttpFS without HDFS config; environments where the same script targets both HDFS and non-HDFS mounts.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/653ec7c26e0fdaa8. Report an issue: GitHub.