apache/hadoop · error · UnsupportedOperationException

does not support listCorruptFileBlocks

Error message

{} does not support listCorruptFileBlocks

What it means

FileSystem.listCorruptFileBlocks(Path) is an optional API: the base implementation in FileSystem.java throws UnsupportedOperationException naming the concrete class. Only HDFS-backed filesystems implement it (Hdfs/DistributedFileSystem ask the NameNode for files whose blocks were found corrupt by DataNode block scanners; FilterFileSystem/FilterFs merely forward to the wrapped FS). The message deliberately names the class so you can see exactly which implementation lacks the capability.

Solutions

1. Run the call only on an HDFS-backed client: guard with fs instanceof DistributedFileSystem
2. Probe the capability first: fs.hasPathCapability(path, CommonPathCapabilities.FS_LIST_CORRUPT_FILE_BLOCKS) (Hadoop 3.3.1+; default probe returns false rather than throwing)
3. Catch UnsupportedOperationException and degrade gracefully (skip corrupt-block reporting for this store) — never retry it
4. For ad-hoc checks use 'hdfs fsck <path> -list-corruptfileblocks' against the HDFS cluster instead of the client API

Example fix

// before
RemoteIterator<Path> corrupt = fs.listCorruptFileBlocks(dir);
// throws UnsupportedOperationException on file://, s3a://, har://

// after
if (fs.hasPathCapability(dir,
        CommonPathCapabilities.FS_LIST_CORRUPT_FILE_BLOCKS)) {
  RemoteIterator<Path> corrupt = fs.listCorruptFileBlocks(dir);
} else {
  LOG.warn("listCorruptFileBlocks unsupported on {}", fs.getUri());
}

import org.apache.hadoop.fs.CommonPathCapabilities;

if (fs.hasPathCapability(dir,
        CommonPathCapabilities.FS_LIST_CORRUPT_FILE_BLOCKS)) {
  RemoteIterator<Path> it = fs.listCorruptFileBlocks(dir);
  // safe to consume
}

static boolean supportsCorruptBlockListing(FileSystem fs) {
  return fs instanceof DistributedFileSystem;
}

try {
  RemoteIterator<Path> it = fs.listCorruptFileBlocks(dir);
} catch (UnsupportedOperationException e) {
  // capability is absent on this implementation: skip, never retry
  LOG.warn("listCorruptFileBlocks unsupported on {}: {}", fs.getUri(), e.getMessage());
}

When it happens

Trigger: Calling fs.listCorruptFileBlocks(path) on any implementation that does not override it: LocalFileSystem/RawLocalFileSystem (file://), HarFileSystem, S3A, ABFS, GCS connectors, or a ViewFileSystem mount over a non-HDFS store. Also reached via FileContext.listCorruptFileBlocks or AbstractFileSystem defaults on such stores.

Common situations: Monitoring/fsck-like tooling written against hdfs:// is repointed at file:// or s3a:// via fs.defaultFS during unit tests or a migration; an ops script enumerates corrupt blocks against the wrong scheme; wrapper filesystems (checksum, view) that delegate to an unsupported inner FS.

Related errors

• does not support method msync
• doesn't support createSnapshot
• doesn't support deleteSnapshot
• doesn't support getAclStatus
• doesn't support getAllStoragePolicies

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/1488a676174adda7. Report an issue: GitHub.

Appendix: source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileSystem.java:2099

      if (filter.accept(listing[i].getPath())) {
        results.add(listing[i]);
      }
    }
  }

  /**
   * List corrupted file blocks.
   *
   * @param path the path.
   * @return an iterator over the corrupt files under the given path
   * (may contain duplicates if a file has more than one corrupt block)
   * @throws UnsupportedOperationException if the operation is unsupported
   *         (default).
   * @throws IOException IO failure
   */
  public RemoteIterator<Path> listCorruptFileBlocks(Path path)
    throws IOException {
    throw new UnsupportedOperationException(getClass().getCanonicalName() +
        " does not support listCorruptFileBlocks");
  }

  /**
   * Filter files/directories in the given path using the user-supplied path
   * filter.
   * <p>
   * Does not guarantee to return the List of files/directories status in a
   * sorted order.
   *
   * @param f
   *          a path name
   * @param filter
   *          the user-supplied path filter
   * @return an array of FileStatus objects for the files under the given path
   *         after applying the filter
   * @throws FileNotFoundException when the path does not exist
   * @throws IOException see specific implementation

View on GitHub (pinned to 2add963021)