apache/hadoop · error · UnsupportedOperationException

Not implemented by the {} FileSystem implementation

Error message

Not implemented by the {} FileSystem implementation

What it means

truncate(Path, newLength) was added in Hadoop 2.7 (HDFS-3107) with no generic implementation: the base FileSystem throws UnsupportedOperationException. Truncation requires block-level recovery, so only HDFS implements it natively; most other filesystems (and older connector versions) fail here.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileSystem.java:1750

   *   <li>Fails if path does not exist.</li>
   *   <li>Fails if path is not closed.</li>
   *   <li>Fails if new size is greater than current size.</li>
   * </ul>
   * @param f The path to the file to be truncated
   * @param newLength The size the file is to be truncated to
   *
   * @return <code>true</code> if the file has been truncated to the desired
   * <code>newLength</code> and is immediately available to be reused for
   * write operations such as <code>append</code>, or
   * <code>false</code> if a background process of adjusting the length of
   * the last block has been started, and clients should wait for it to
   * complete before proceeding with further file updates.
   * @throws IOException IO failure
   * @throws UnsupportedOperationException if the operation is unsupported
   *         (default).
   */
  public boolean truncate(Path f, long newLength) throws IOException {
    throw new UnsupportedOperationException("Not implemented by the " +
        getClass().getSimpleName() + " FileSystem implementation");
  }

  /**
   * Delete a file/directory.
   * @param f the path.
   * @throws IOException IO failure.
   * @return if delete success true, not false.
   * @deprecated Use {@link #delete(Path, boolean)} instead.
   */
  @Deprecated
  public boolean delete(Path f) throws IOException {
    return delete(f, true);
  }

  /** Delete a file.
   *
   * @param f the path to delete.

View on GitHub (pinned to 2add963021)

Solutions

1. Gate on implementation: only truncate when fs instanceof DistributedFileSystem
2. Emulate for other stores: copy the first newLength bytes to a temp file, delete the original, rename temp back
3. Upgrade the connector/Hadoop if a newer version implements truncate for your store

Example fix

// before
fs.truncate(path, newLen); // UnsupportedOperationException

// after
if (fs instanceof DistributedFileSystem) {
  fs.truncate(path, newLen);
} else {
  Path tmp = path.suffix(".trunc");
  byte[] buf = new byte[8192];
  try (FSDataInputStream in = fs.open(path);
       FSDataOutputStream out = fs.create(tmp, true)) {
    long remaining = newLen;
    while (remaining > 0) {
      int n = in.read(buf, 0, (int) Math.min(buf.length, remaining));
      if (n < 0) break;
      out.write(buf, 0, n);
      remaining -= n;
    }
  }
  fs.delete(path, false);
  fs.rename(tmp, path);
}

public static boolean canTruncateInPlace(FileSystem fs) {
  return fs instanceof DistributedFileSystem;
}

try {
  fs.truncate(path, newLen);
} catch (UnsupportedOperationException e) {
  truncateByCopy(fs, path, newLen); // copy prefix, delete, rename back
}

When it happens

Trigger: Calling fs.truncate(path, len) on a non-HDFS FileSystem or on an implementation built from an older Hadoop that lacks the override (old local FS, har://, various connectors).

Common situations: Applications written against HDFS truncate semantics and ported to S3/local storage; libraries using truncate to maintain fixed-size record files run in local tests.

Related errors

• {getClass().getSimpleName()} doesn't support truncate
• Truncate is not supported by ChecksumFileSystem
• Truncate is not supported by ChecksumFs
• Dest filesystem '${fs.getUri().getScheme()}' doesn't support
• {} doesn't support satisfyStoragePolicy

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/9a61c2adbb048a46. Report an issue: GitHub.