{"record":{"id":"e5bb5d922dcdacf5","repo":"apache/hadoop","slug":"failed-to-put-file-since-rename-fail","errorCode":null,"errorMessage":"failed to put file since rename fail.","messagePattern":"failed to put file since rename fail\\.","errorType":"exception","errorClass":"RuntimeException","httpStatus":null,"severity":"error","filePath":"hadoop-cloud-storage-project/hadoop-tos/src/main/java/org/apache/hadoop/fs/tosfs/object/FileStore.java","lineNumber":399,"sourceCode":"  private static void copyInputStreamToFile(InputStream in, File partFile, long contentLength) {\n    File tmpFile = createTmpFile(partFile);\n    try (FileOutputStream out = new FileOutputStream(tmpFile)) {\n      long copiedBytes = IOUtils.copyLarge(in, out, 0, contentLength);\n\n      if (copiedBytes < contentLength) {\n        throw new IOException(\n            String.format(\"Unexpect end of stream, expected length:%s, actual:%s\", contentLength,\n                tmpFile.length()));\n      }\n    } catch (IOException e) {\n      CommonUtils.runQuietly(() -> FileUtils.delete(tmpFile));\n      throw new RuntimeException(e);\n    } finally {\n      CommonUtils.runQuietly(in::close);\n    }\n\n    if (!tmpFile.renameTo(partFile)) {\n      throw new RuntimeException(\"failed to put file since rename fail.\");\n    }\n  }\n\n  @Override\n  public byte[] completeUpload(String key, String uploadId, List<Part> uploadParts) {\n    Preconditions.checkArgument(uploadParts != null && uploadParts.size() > 0,\n        \"upload parts cannot be null or empty.\");\n    File uploadDir = uploadPath(key, uploadId).toFile();\n    if (!uploadDir.exists()) {\n      throw new RuntimeException(\"cannot locate the upload id: \" + uploadId);\n    }\n\n    List<Integer> partNums = listPartNums(uploadDir);\n    if (partNums.size() != uploadParts.size()) {\n      throw new RuntimeException(String.format(\"parts length mismatched: %d != %d\",\n          partNums.size(), uploadParts.size()));\n    }\n","sourceCodeStart":381,"sourceCodeEnd":417,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-cloud-storage-project/hadoop-tos/src/main/java/org/apache/hadoop/fs/tosfs/object/FileStore.java#L381-L417","documentation":"After FileStore.copyInputStreamToFile() successfully writes a staged .tmp.<uuid> file, it promotes it to the destination with File.renameTo(); if the rename returns false it throws 'failed to put file since rename fail.' The copy succeeded but the atomic swap did not, so the destination never receives the data. renameTo() returning false on the same directory usually means permission trouble, a file lock on the target, or platform-specific rename restrictions.","triggerScenarios":"put() or uploadPart() where renameTo(partFile) fails: destination file locked by a concurrent reader (Windows/antivirus), no write permission on the directory, a stale destination file with restrictive permissions, or the tmp/destination straddling a weird mount inside the store root.","commonSituations":"Running tests on Windows while another process (AV scanner, another test thread) holds the part file open; store root placed on a filesystem with permission quirks (NFS, some container mounts); leftover read-only part files from a previous run owned by another user.","solutions":["Check directory permissions on the store root and staging tree: they must be writable by the process user (chmod, chown)","On Windows or when a target already exists, delete the destination file first and retry the operation (Linux silently replaces; Windows refuses)","Move the store root off exotic mounts (NFS/FUSE) onto local disk (tmpfs/ext4)","Remove stale .tmp.* and part files from crashed runs inside __STAGING__ so renames never collide with locked files"],"exampleFix":null,"handlingStrategy":"validation","validationCode":"java.nio.file.Path dir = java.nio.file.Paths.get(root);\nif (!java.nio.file.Files.isWritable(dir)) {\n  throw new IOException(\"store root not writable, renames will fail: \" + dir);\n}\n// avoid locked destinations before multipart work on Windows\nif (storage.objectStatus(key) != null && isWindows()) {\n  storage.delete(key);\n}","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Ensure the process user owns or can write every directory under the store root","Avoid concurrent readers on keys being rewritten when running on Windows (file locks block renameTo)","Keep the store root off NFS/FUSE mounts; use local disk or tmpfs","Clean leftover .tmp.* and part files after crashed runs so renames never collide"],"tags":["hadoop","tos","filestore","rename","local-storage"],"backgroundTag":"file-rename-failed","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}