{"record":{"id":"592cc69fd968ffaf","repo":"apache/iceberg","slug":"failed-to-delete-staging-file-592cc6","errorCode":null,"errorMessage":"Failed to delete staging file: {}","messagePattern":"Failed to delete staging file: (.+?)","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"aws/src/main/java/org/apache/iceberg/aws/s3/S3OutputStream.java","lineNumber":339,"sourceCode":"              UploadPartRequest uploadRequest = requestBuilder.build();\n\n              CompletableFuture<CompletedPart> future =\n                  CompletableFuture.supplyAsync(\n                          () -> {\n                            UploadPartResponse response =\n                                s3.uploadPart(uploadRequest, RequestBody.fromFile(f));\n                            return CompletedPart.builder()\n                                .eTag(response.eTag())\n                                .partNumber(uploadRequest.partNumber())\n                                .build();\n                          },\n                          executorService)\n                      .whenComplete(\n                          (result, thrown) -> {\n                            try {\n                              Files.deleteIfExists(f.toPath());\n                            } catch (IOException e) {\n                              LOG.warn(\"Failed to delete staging file: {}\", f, e);\n                            }\n\n                            if (thrown != null) {\n                              // Exception observed here will be thrown as part of\n                              // CompletionException\n                              // when we will join completable futures.\n                              LOG.error(\"Failed to upload part: {}\", uploadRequest, thrown);\n                            }\n                          });\n\n              multiPartMap.put(f, future);\n            });\n  }\n\n  private void completeMultiPartUpload() {\n    Preconditions.checkState(closed, \"Complete upload called on open stream: \" + location);\n\n    List<CompletedPart> completedParts;","sourceCodeStart":321,"sourceCodeEnd":357,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/aws/src/main/java/org/apache/iceberg/aws/s3/S3OutputStream.java#L321-L357","documentation":"S3OutputStream buffers data to local staging files, then uploads them as multipart parts asynchronously. After each part upload completes, the staging file is deleted; if that filesystem deletion throws IOException, it logs this warning with the file path and exception. Upload results are unaffected, but staging files can accumulate in the temp directory.","triggerScenarios":"Writing an S3 output file where the temp staging directory's file cannot be deleted — read-only or full temp filesystem, file locked by another process, or permissions changed mid-run.","commonSituations":"Containers with small/full /tmp volumes; concurrent uploads on NFS-mounted temp dirs; jobs running under restricted users whose tmpdir permissions drift.","solutions":["Ensure the temp directory (java.io.tmpdir / s3.staging-directory) is writable and has free space","Manually clean leftover staging files from previous runs","Point the staging directory to a reliable local disk instead of NFS","Check the attached IOException for the exact OS-level reason"],"exampleFix":"// before\nprops.put(\"s3.staging-directory\", \"/mnt/nfs/tmp\");\n// after: local disk with adequate space\nprops.put(\"s3.staging-directory\", \"/var/tmp/iceberg-staging\");","handlingStrategy":"validation","validationCode":"// before writing, verify staging dir is usable\nFile dir = new File(System.getProperty(\"java.io.tmpdir\"));\nFile probe = File.createTempFile(\"iceberg-probe\", null, dir); probe.deleteOnExit();","typeGuard":null,"tryCatchPattern":"try (OutputStream out = outputFile.create()) { write(out); } // staging cleanup failures are warnings; check logs for leftover files","preventionTips":["Ensure java.io.tmpdir (or configured staging directory) is writable local disk with free space","Avoid NFS mounts for staging; use instance-local storage","Periodically sweep the staging directory for orphaned files after job crashes"],"tags":["aws","s3","filesystem","cleanup"],"backgroundTag":"file-write-failed","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}