{"record":{"id":"708f523531e9c35b","repo":"apache/hadoop","slug":"failed-to-copy-full-contents-from-srcfile-to","errorCode":null,"errorMessage":"Failed to copy full contents from '{srcFile}' to '{destFile}'","messagePattern":"Failed to copy full contents from '(.+?)' to '(.+?)'","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/Storage.java","lineNumber":1369,"sourceCode":"        throw new IOException(\"Destination '\" + destFile\n            + \"' exists but is read-only\");\n      } else {\n        if (destFile.delete() == false) {\n          throw new IOException(\"Destination '\" + destFile\n              + \"' exists but cannot be deleted\");\n        }\n      }\n    }\n    try {\n      NativeIO.copyFileUnbuffered(srcFile, destFile);\n    } catch (NativeIOException e) {\n      throw new IOException(\"Failed to copy \" + srcFile.getCanonicalPath()\n          + \" to \" + destFile.getCanonicalPath()\n          + \" due to failure in NativeIO#copyFileUnbuffered(). \"\n          + e.toString());\n    }\n    if (srcFile.length() != destFile.length()) {\n      throw new IOException(\"Failed to copy full contents from '\" + srcFile\n          + \"' to '\" + destFile + \"'\");\n    }\n    if (preserveFileDate) {\n      if (destFile.setLastModified(srcFile.lastModified()) == false) {\n        LOG.debug(\"Failed to preserve last modified date from'{}' to '{}'\",\n            srcFile, destFile);\n      }\n    }\n  }\n\n  /**\n   * Recursively delete all the content of the directory first and then \n   * the directory itself from the local filesystem.\n   * @param dir The directory to delete\n   * @throws IOException\n   */\n  public static void deleteDir(File dir) throws IOException {\n    if (!FileUtil.fullyDelete(dir))","sourceCodeStart":1351,"sourceCodeEnd":1387,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/Storage.java#L1351-L1387","documentation":"Post-copy integrity check of Storage.nativeCopyFileUnbuffered: the native copy returned without error, but srcFile.length() != destFile.length(), i.e. the destination is truncated. This catches partial writes the syscall did not report - most often because the source was still being written while copied, or the write side hit a quota/space limit silently.","triggerScenarios":"Copying a file that is still being appended (checkpoint not finalized); destination quota or sparse-file behavior truncating the write; NFS oddities in size reporting; source replaced mid-copy.","commonSituations":"Checkpoint race - copying fsimage before the NameNode finalized it; quota-limited volumes; flaky network filesystems for storage dirs.","solutions":["Wait for the writer to finish (e.g. checkpoint finalized) and confirm the source size is stable across two stats, then retry the copy","Check destination volume space and quotas","If it recurs on local disk, suspect hardware and inspect dmesg/SMART"],"exampleFix":null,"handlingStrategy":"retry","validationCode":"long l1 = srcFile.length();\nThread.sleep(1000);\nlong l2 = srcFile.length();\nif (l1 != l2) throw new IOException(\"source still being written: \" + srcFile);","typeGuard":null,"tryCatchPattern":"try {\n  Storage.nativeCopyFileUnbuffered(src, dst, true);\n} catch (IOException e) {\n  if (e.getMessage() != null && e.getMessage().contains(\"Failed to copy full contents\")) {\n    // source was mid-write: wait for the producer to finalize, then retry the copy\n  }\n}","preventionTips":["Only copy files after the producer signals completion (checkpoint finalized, not merely visible)","Verify source size is stable across two stats before copying","Check destination quotas as part of pre-copy validation"],"tags":["hdfs","file-copy","data-integrity","truncated-file","race-condition"],"backgroundTag":"partial-file-copy","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}