{"record":{"id":"8428c526267c53c4","repo":"apache/hadoop","slug":"truncate-is-not-supported-by-checksumfs","errorCode":null,"errorMessage":"Truncate is not supported by ChecksumFs","messagePattern":"Truncate is not supported by ChecksumFs","errorType":"exception","errorClass":"UnsupportedOperationException","httpStatus":null,"severity":"error","filePath":"hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/ChecksumFs.java","lineNumber":325,"sourceCode":"     *\n     * @param      pos   the postion to seek to.\n     * @exception  IOException  if an I/O error occurs or seeks after EOF\n     *             ChecksumException if the chunk to seek to is corrupted\n     */\n\n    @Override\n    public synchronized void seek(long pos) throws IOException { \n      if (pos>getFileLength()) {\n        throw new IOException(\"Cannot seek after EOF\");\n      }\n      super.seek(pos);\n    }\n\n  }\n\n  @Override\n  public boolean truncate(Path f, long newLength) throws IOException {\n    throw new UnsupportedOperationException(\"Truncate is not supported \"\n        + \"by ChecksumFs\");\n  }\n\n  /**\n   * Opens an FSDataInputStream at the indicated Path.\n   * @param f the file name to open\n   * @param bufferSize the size of the buffer to be used.\n   */\n  @Override\n  public FSDataInputStream open(Path f, int bufferSize) \n    throws IOException, UnresolvedLinkException {\n    return new FSDataInputStream(\n        new ChecksumFSInputChecker(this, f, bufferSize));\n  }\n\n  /**\n   * Calculated the length of the checksum file in bytes.\n   * @param size the length of the data file in bytes","sourceCodeStart":307,"sourceCodeEnd":343,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/ChecksumFs.java#L307-L343","documentation":"ChecksumFs is the AbstractFileSystem-side checksumming wrapper (FileContext on file:// resolves to LocalFs, a ChecksumFs subclass). Like its FileSystem twin, it cannot keep the sidecar .crc consistent across a truncate, so truncate(Path, long) always throws UnsupportedOperationException. Truncation must bypass the checksum layer via getRawFs() (line 78 of ChecksumFs.java) or be done as a rewrite.","triggerScenarios":"Calling FileContext.truncate(f, newLength) (FileContext.java:947 -> AbstractFileSystem.truncate -> ChecksumFs.truncate at line 324) when the default fs or the resolved AbstractFileSystem is a ChecksumFs, i.e., any truncate on file:/// paths through FileContext.","commonSituations":"Applications using the FileContext API (as advised for new code) reusing truncate logic proven on HDFS against local test fixtures; cross-fs utilities that call fc.truncate unconditionally.","solutions":["Truncate the raw filesystem: ((ChecksumFs) fc.getDefaultFileSystem()).getRawFs().truncate(f, newLength) - RawLocalFileSystem.truncate is implemented - then delete the stale sidecar .crc.","Rewrite: copy the first newLength bytes to a temp file, fs.rename it over the target; a fresh .crc is generated on the next create.","In portable code, catch UnsupportedOperationException from truncate and fall back to copy-truncate-rename."],"exampleFix":"// before\nFileContext fc = FileContext.getLocalFSFileContext();\nfc.truncate(path, 1024); // throws UnsupportedOperationException\n\n// after\nChecksumFs cfs = (ChecksumFs) fc.getDefaultFileSystem();\ncfs.getRawFs().truncate(path, 1024);\njava.nio.file.Files.deleteIfExists(\n    java.nio.file.Paths.get(path.getParent().toString(), \".\" + path.getName() + \".crc\"));","handlingStrategy":"validation","validationCode":"AbstractFileSystem afs = fc.getDefaultFileSystem();\nif (afs instanceof ChecksumFs) {\n  // truncate unsupported: use ((ChecksumFs) afs).getRawFs().truncate + drop .crc, or rewrite\n}","typeGuard":"static boolean canTruncateFc(FileContext fc) throws IOException {\n  return !(fc.getDefaultFileSystem() instanceof ChecksumFs);\n}","tryCatchPattern":"try {\n  fc.truncate(path, newLen);\n} catch (UnsupportedOperationException e) {\n  ((ChecksumFs) fc.getDefaultFileSystem()).getRawFs().truncate(path, newLen);\n  // plus delete the stale .crc sidecar\n}","preventionTips":["Do not assume FileContext.truncate is universal; probe the AbstractFileSystem type on file://.","After raw truncate, delete the .crc sidecar to keep future verified reads valid.","Cover truncate in file://-based tests for any portable fs code."],"tags":["hadoop","filecontext","filesystem","truncate","local-filesystem","unsupported-operation"],"backgroundTag":"unsupported-filesystem-operation","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}