apache/hadoop · error · PathIOException

Not an abfs filesystem: %s

Error message

Not an abfs filesystem: %s

What it means

AbfsManifestStoreOperations implements the manifest-store binding for the ABFS magic committer (uncommitted-file reconciliation). bindToFileSystem requires an AzureBlobFileSystem instance; any other FileSystem implementation (RawLocalFileSystem, s3a, gcs, viewfs, wasb) triggers PathIOException 'Not an abfs filesystem: <class>' before any commit work starts. This is a wiring/binding error, not a data problem.

Source

Thrown at hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/commit/AbfsManifestStoreOperations.java:85

   */
  private boolean etagsPreserved;

  @Override
  public AzureBlobFileSystem getFileSystem() {
    return (AzureBlobFileSystem) super.getFileSystem();
  }

  /**
   * Bind to the store.
   *
   * @param filesystem FS.
   * @param path path to work under
   * @throws IOException binding problems.
   */
  @Override
  public void bindToFileSystem(FileSystem filesystem, Path path) throws IOException {
    if (!(filesystem instanceof AzureBlobFileSystem)) {
      throw new PathIOException(path.toString(),
          "Not an abfs filesystem: " + filesystem.getClass());
    }
    super.bindToFileSystem(filesystem, path);
    try {
      resilientCommitByRename = getFileSystem().createResilientCommitSupport(path);
      // this also means that etags are preserved.
      etagsPreserved = true;
      LOG.debug("Bonded to filesystem with resilient commits under path {}", path);
    } catch (UnsupportedOperationException e) {
      LOG.debug("No resilient commit support under path {}", path);
    }
  }

  /**
   * Etags are preserved through Gen2 stores, but not wasb stores.
   * @param path path to probe.
   * @return true if this store preserves etags.
   */

View on GitHub (pinned to 2add963021)

Solutions

1. Bind the ABFS manifest committer only to abfs(s):// output paths; select committers per scheme.
2. Scope mapreduce.outputcommitter.factory.scheme.abfs / .abfss keys so they do not attach to file:// or other schemes.
3. In tests, either bind to a real AzureBlobFileSystem or use the committer implementation matching the local filesystem.
4. Verify fs.getUri().getScheme() before constructing the manifest store.

Example fix

// before
ManifestStoreOperations store = new AbfsManifestStoreOperations();
store.bindToFileSystem(localFs, outputPath);   // throws

// after
if (!(fs instanceof AzureBlobFileSystem)) {
  throw new IOException("ABFS manifest committer requires an abfs:// path, got: " + fs.getUri());
}
store.bindToFileSystem(fs, outputPath);

if (!(fs instanceof AzureBlobFileSystem)) {
  throw new IOException("ABFS manifest committer requires abfs://, got: " + fs.getUri());
}

static boolean isAbfsFileSystem(FileSystem fs) {
  return fs instanceof org.apache.hadoop.fs.azurebfs.AzureBlobFileSystem;
}

try {
  storeOps.bindToFileSystem(fs, path);
} catch (PathIOException e) {
  if (e.getMessage().startsWith("Not an abfs filesystem")) { /* select a committer matching fs.getUri().getScheme() */ }
}

When it happens

Trigger: Constructing ManifestCommitOperations with AbfsManifestStoreOperations and calling bindToFileSystem on a non-ABFS filesystem - most often the output path resolving to file:/// in tests, or committer factories applied globally so some tasks bind to local/other filesystems.

Common situations: Committer configuration (mapreduce.outputcommitter.factory.scheme.*) applied to jobs whose working/output paths are local; unit tests binding the ABFS manifest store to a mock or local FS; copy-paste of ABFS committer settings into non-ABFS pipelines.

Related errors

• WASB Driver using wasb(s) schema is no longer supported. Ins
• "%s" must be set for user-bound SAS auth type.
• Unable to load user-bound SAS token provider class: {e}
• ABFS endpoint is not set correctly : %s, Do not specify sche
• EncryptionContext not present in GetPathStatus response

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/550c0eadb19eb602. Report an issue: GitHub.