{"record":{"id":"8ad7a56c4cc8af98","repo":"apache/hadoop","slug":"not-a-file-s","errorCode":null,"errorMessage":"Not a file: %s","messagePattern":"Not a file: (.+?)","errorType":"exception","errorClass":"FileNotFoundException","httpStatus":null,"severity":"error","filePath":"hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/commit/impl/CommitOperations.java","lineNumber":533,"sourceCode":"   * @param destPath destination path\n   * @param partition partition/subdir. Not used\n   * @param uploadPartSize size of upload\n   * @param progress progress callback\n   * @return a pending upload entry\n   * @throws IOException failure\n   */\n  public SinglePendingCommit uploadFileToPendingCommit(File localFile,\n      Path destPath,\n      String partition,\n      long uploadPartSize,\n      Progressable progress)\n      throws IOException {\n\n    LOG.debug(\"Initiating multipart upload from {} to {}\",\n        localFile, destPath);\n    Preconditions.checkArgument(destPath != null);\n    if (!localFile.isFile()) {\n      throw new FileNotFoundException(\"Not a file: \" + localFile);\n    }\n    String destURI = destPath.toUri().toString();\n    String destKey = fs.pathToKey(destPath);\n    String uploadId = null;\n\n    // flag to indicate to the finally clause that the operation\n    // failed. it is cleared as the last action in the try block.\n    boolean threw = true;\n    final DurationTracker tracker = statistics.trackDuration(\n        COMMITTER_STAGE_FILE_UPLOAD.getSymbol());\n    try (DurationInfo d = new DurationInfo(LOG,\n        \"Upload staged file from %s to %s\",\n        localFile.getAbsolutePath(),\n        destPath)) {\n\n      statistics.commitCreated();\n      uploadId = writeOperations.initiateMultiPartUpload(destKey,\n          PutObjectOptions.defaultOptions());","sourceCodeStart":515,"sourceCodeEnd":551,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/commit/impl/CommitOperations.java#L515-L551","documentation":"FileNotFoundException from CommitOperations.uploadFileToPendingCommit when the local file to upload does not exist as a regular file (localFile.isFile() is false). This is the entry point that turns a locally staged task output into a multipart upload plus a SinglePendingCommit record, so the local file must still be present and readable at commit time. The check runs after the 'Initiating multipart upload' debug log and before any S3 call.","triggerScenarios":"uploadFileToPendingCommit is called with a File that was deleted (task attempt working directory cleaned up) or that names a directory rather than a file.","commonSituations":"NodeManager local directories are cleaned (disk-policy eviction or yarn.nodemanager.local-dirs housekeeping) before job commit; a task retry removed the previous attempt's files; someone manually deleted the staging directory; the path points at the attempt directory itself instead of a file inside it.","solutions":["Confirm the staged file still exists and is a regular file before the commit phase (File#isFile check in a precommit hook)","Keep task attempt directories intact until job commit finishes: check NodeManager disk utilization and local-dirs cleanup policy on the node that ran the task","If files were lost, rerun the failed job rather than trying to hand-craft pending commit records"],"exampleFix":"// before\ncommitData = ops.uploadFileToPendingCommit(localFile, destPath, partition, partSize, progress);\n\n// after\nif (!localFile.isFile()) {\n  throw new IOException(\"Staged file missing before commit: \" + localFile);\n}\ncommitData = ops.uploadFileToPendingCommit(localFile, destPath, partition, partSize, progress);","handlingStrategy":"validation","validationCode":"if (!localFile.isFile() || !localFile.canRead()) {\n  throw new IOException(\"Staged output missing/unreadable before commit: \"\n      + localFile.getAbsolutePath());\n}","typeGuard":null,"tryCatchPattern":"try {\n  SinglePendingCommit data = ops.uploadFileToPendingCommit(\n      localFile, destPath, partition, partSize, progress);\n} catch (FileNotFoundException e) {\n  // local staging file vanished: fail the task so the attempt reruns on a healthy node\n  LOG.error(\"Staged file lost before upload: {}\", localFile, e);\n  throw e;\n}","preventionTips":["Monitor NodeManager local-dirs disk usage so eviction cannot delete live attempt directories","Never clean task attempt directories before job commit completes","In precommit hooks, verify every expected staged file exists before letting commitJob start"],"tags":["s3a","committer","local-file","staging"],"backgroundTag":"missing-local-file","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}