apache/hadoop · error · UnsupportedOperationException

{} does not support method msync

Error message

{} does not support method msync

What it means

FileSystem.mync() synchronizes the client's metadata state so subsequent reads observe the latest namespace/block state — needed for consistent reads in HDFS HA (HDFS-13786). The base FileSystem throws UnsupportedOperationException; only HDFS clients (DistributedFileSystem, Hdfs) and pass-through wrappers (FilterFileSystem delegates; HarFileSystem no-ops) implement it. Local filesystem and object-store connectors reject the call entirely.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileSystem.java:2808

   * Return a file status object that represents the path.
   * @param f The path we want information from
   * @return a FileStatus object
   * @throws FileNotFoundException when the path does not exist
   * @throws IOException see specific implementation
   */
  public abstract FileStatus getFileStatus(Path f) throws IOException;

  /**
   * Synchronize client metadata state.
   * <p>
   * In some FileSystem implementations such as HDFS metadata
   * synchronization is essential to guarantee consistency of read requests
   * particularly in HA setting.
   * @throws IOException If an I/O error occurred.
   * @throws UnsupportedOperationException if the operation is unsupported.
   */
  public void msync() throws IOException, UnsupportedOperationException {
    throw new UnsupportedOperationException(getClass().getCanonicalName() +
        " does not support method msync");
  }

  /**
   * Checks if the user can access a path.  The mode specifies which access
   * checks to perform.  If the requested permissions are granted, then the
   * method returns normally.  If access is denied, then the method throws an
   * {@link AccessControlException}.
   * <p>
   * The default implementation calls {@link #getFileStatus(Path)}
   * and checks the returned permissions against the requested permissions.
   *
   * Note that the {@link #getFileStatus(Path)} call will be subject to
   * authorization checks.
   * Typically, this requires search (execute) permissions on each directory in
   * the path's prefix, but this is implementation-defined.  Any file system
   * that provides a richer authorization model (such as ACLs) may override the
   * default implementation so that it checks against that model instead.

View on GitHub (pinned to 2add963021)

Solutions

  1. Guard with fs instanceof DistributedFileSystem before calling msync()
  2. Catch UnsupportedOperationException and continue: on stores without msync, reads are either already consistent or the call is meaningless — do not retry
  3. If you own the FileSystem implementation, override msync() as a no-op when the store is already consistent (the HarFileSystem pattern)

Example fix

// before
fs.msync(); // throws on LocalFileSystem/S3A/...
Path p = new Path("/flag");

// after
if (fs instanceof DistributedFileSystem) {
  fs.msync(); // only meaningful for HDFS HA consistency
}
Path p = new Path("/flag");
Defensive patterns

Strategy: try-catch

Validate before calling

if (fs instanceof DistributedFileSystem) {
  fs.msync(); // only HDFS implements metadata sync
}

Type guard

static boolean supportsMsync(FileSystem fs) {
  return fs instanceof DistributedFileSystem;
}

Try / catch

try {
  fs.msync();
} catch (UnsupportedOperationException e) {
  // store is already consistent or has no msync concept: continue
}

Prevention

When it happens

Trigger: Calling fs.msync() on LocalFileSystem (RawLocalFileSystem does not override it), S3A, ABFS, GCS, or a custom FileSystem that did not override the method; calling FileContext.msync() over such a store (AbstractFileSystem.msync has the same throwing default).

Common situations: Read-your-writes polling loops written for an HDFS HA cluster are run unchanged in local unit tests or against object stores; shared library code calls msync() unconditionally after open()/getFileStatus() to force consistency.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/a31005e13c506269. Report an issue: GitHub.