apache/hadoop · error · UnsupportedOperationException

Truncate is not supported by ChecksumFs

Error message

Truncate is not supported by ChecksumFs

What it means

ChecksumFs is the AbstractFileSystem-side checksumming wrapper (FileContext on file:// resolves to LocalFs, a ChecksumFs subclass). Like its FileSystem twin, it cannot keep the sidecar .crc consistent across a truncate, so truncate(Path, long) always throws UnsupportedOperationException. Truncation must bypass the checksum layer via getRawFs() (line 78 of ChecksumFs.java) or be done as a rewrite.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/ChecksumFs.java:325

     *
     * @param      pos   the postion to seek to.
     * @exception  IOException  if an I/O error occurs or seeks after EOF
     *             ChecksumException if the chunk to seek to is corrupted
     */

    @Override
    public synchronized void seek(long pos) throws IOException { 
      if (pos>getFileLength()) {
        throw new IOException("Cannot seek after EOF");
      }
      super.seek(pos);
    }

  }

  @Override
  public boolean truncate(Path f, long newLength) throws IOException {
    throw new UnsupportedOperationException("Truncate is not supported "
        + "by ChecksumFs");
  }

  /**
   * Opens an FSDataInputStream at the indicated Path.
   * @param f the file name to open
   * @param bufferSize the size of the buffer to be used.
   */
  @Override
  public FSDataInputStream open(Path f, int bufferSize) 
    throws IOException, UnresolvedLinkException {
    return new FSDataInputStream(
        new ChecksumFSInputChecker(this, f, bufferSize));
  }

  /**
   * Calculated the length of the checksum file in bytes.
   * @param size the length of the data file in bytes

View on GitHub (pinned to 2add963021)

Solutions

1. Truncate the raw filesystem: ((ChecksumFs) fc.getDefaultFileSystem()).getRawFs().truncate(f, newLength) - RawLocalFileSystem.truncate is implemented - then delete the stale sidecar .crc.
2. Rewrite: copy the first newLength bytes to a temp file, fs.rename it over the target; a fresh .crc is generated on the next create.
3. In portable code, catch UnsupportedOperationException from truncate and fall back to copy-truncate-rename.

Example fix

// before
FileContext fc = FileContext.getLocalFSFileContext();
fc.truncate(path, 1024); // throws UnsupportedOperationException

// after
ChecksumFs cfs = (ChecksumFs) fc.getDefaultFileSystem();
cfs.getRawFs().truncate(path, 1024);
java.nio.file.Files.deleteIfExists(
    java.nio.file.Paths.get(path.getParent().toString(), "." + path.getName() + ".crc"));

AbstractFileSystem afs = fc.getDefaultFileSystem();
if (afs instanceof ChecksumFs) {
  // truncate unsupported: use ((ChecksumFs) afs).getRawFs().truncate + drop .crc, or rewrite
}

static boolean canTruncateFc(FileContext fc) throws IOException {
  return !(fc.getDefaultFileSystem() instanceof ChecksumFs);
}

try {
  fc.truncate(path, newLen);
} catch (UnsupportedOperationException e) {
  ((ChecksumFs) fc.getDefaultFileSystem()).getRawFs().truncate(path, newLen);
  // plus delete the stale .crc sidecar
}

When it happens

Trigger: Calling FileContext.truncate(f, newLength) (FileContext.java:947 -> AbstractFileSystem.truncate -> ChecksumFs.truncate at line 324) when the default fs or the resolved AbstractFileSystem is a ChecksumFs, i.e., any truncate on file:/// paths through FileContext.

Common situations: Applications using the FileContext API (as advised for new code) reusing truncate logic proven on HDFS against local test fixtures; cross-fs utilities that call fc.truncate unconditionally.

Related errors

• Truncate is not supported by ChecksumFileSystem
• {getClass().getSimpleName()} doesn't support truncate
• Append is not supported by ChecksumFileSystem
• Concat is not supported by ChecksumFileSystem
• Not implemented by the {} FileSystem implementation

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/8428c526267c53c4. Report an issue: GitHub.