{"record":{"id":"97e73e9c8c6f18be","repo":"apache/hadoop","slug":"part-num-mismatched-d-d","errorCode":null,"errorMessage":"part num mismatched: %d != %d","messagePattern":"part num mismatched: (.+?) != (.+?)","errorType":"exception","errorClass":"RuntimeException","httpStatus":null,"severity":"error","filePath":"hadoop-cloud-storage-project/hadoop-tos/src/main/java/org/apache/hadoop/fs/tosfs/object/FileStore.java","lineNumber":429,"sourceCode":"\n    List<Integer> partNums = listPartNums(uploadDir);\n    if (partNums.size() != uploadParts.size()) {\n      throw new RuntimeException(String.format(\"parts length mismatched: %d != %d\",\n          partNums.size(), uploadParts.size()));\n    }\n\n    Collections.sort(partNums);\n    uploadParts.sort(Comparator.comparingInt(Part::num));\n\n    Path keyPath = path(encode(key));\n    File tmpFile = createTmpFile(keyPath.toFile());\n    try (FileOutputStream outputStream = new FileOutputStream(tmpFile);\n        FileChannel outputChannel = outputStream.getChannel()) {\n      int offset = 0;\n      for (int i = 0; i < partNums.size(); i++) {\n        Part part = uploadParts.get(i);\n        if (part.num() != partNums.get(i)) {\n          throw new RuntimeException(\n              String.format(\"part num mismatched: %d != %d\", part.num(), partNums.get(i)));\n        }\n\n        File partFile = new File(uploadDir, String.valueOf(part.num()));\n        checkPartFile(part, partFile);\n\n        try (FileInputStream inputStream = new FileInputStream(partFile);\n            FileChannel inputChannel = inputStream.getChannel()) {\n          outputChannel.transferFrom(inputChannel, offset, partFile.length());\n          offset += partFile.length();\n        }\n      }\n    } catch (IOException e) {\n      throw new RuntimeException(e);\n    }\n\n    if (!tmpFile.renameTo(keyPath.toFile())) {\n      throw new RuntimeException(\"rename file failed\");","sourceCodeStart":411,"sourceCodeEnd":447,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-cloud-storage-project/hadoop-tos/src/main/java/org/apache/hadoop/fs/tosfs/object/FileStore.java#L411-L447","documentation":"FileStore.completeUpload() sorts the staged part-file numbers and the caller's uploadParts by part number, then walks them pairwise; if part.num() differs from the i-th staged part number it throws 'part num mismatched'. The set of part numbers on disk and the set in the request are not identical (gap, duplicate, or shifted numbering), even though the counts matched.","triggerScenarios":"completeUpload where part numbers were re-assigned between upload and complete (e.g. re-uploading with different numbering), a part file was renamed/removed and a later part renumbered to fill the gap, or the caller reorders/renumbers its Part list (uploadPartCopy results spliced into an uploadPart sequence with clashing numbers).","commonSituations":"Mixing uploadPart and uploadPartCopy results with independently chosen part numbers; recovery code that re-numbers parts; tests that stage part files manually with names like 0, 1, 2 (listPartNums reads the file names as the numbers, and part numbers must be a strictly matching sorted sequence).","solutions":["Assign each part number exactly once at upload time and never renumber: numbers on disk must equal the sorted numbers in your parts list","Feed completeUpload the untouched List<Part> returned by the upload calls","If any part failed or numbering diverged, abort the upload and redo all parts under a new uploadId with fresh 1..N numbering","Never write files into __STAGING__/<key>/<uploadId> yourself — file names are the part numbers"],"exampleFix":"// before: renumbered parts between upload and complete\nList<Part> shifted = parts.stream().map(p -> new Part(p.num() - 1, p.size(), p.eTag())).collect(toList());\nstorage.completeUpload(key, uploadId, shifted); // part num mismatched: 0 != 1\n// after: pass the parts exactly as returned by uploadPart\nstorage.completeUpload(key, uploadId, partsFromUploadCalls);","handlingStrategy":"validation","validationCode":"List<Integer> nums = parts.stream().map(Part::num).sorted()\n    .collect(java.util.stream.Collectors.toList());\nfor (int i = 0; i < nums.size(); i++) {\n  if (nums.get(i) != i + 1) {\n    throw new IOException(\"part numbers must be exactly 1..N; got \" + nums);\n  }\n}\nstorage.completeUpload(key, uploadId, parts);","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Assign part numbers once at upload time and never renumber between upload and complete","Feed the untouched List<Part> from the upload calls straight into completeUpload","When mixing uploadPart and uploadPartCopy, allocate part numbers from one counter to avoid clashes","Remember staging file names ARE the part numbers: never rename or add files there"],"tags":["hadoop","tos","filestore","multipart-upload","data-integrity"],"backgroundTag":"multipart-parts-mismatch","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-23T01:17:44.959Z"}