{"record":{"id":"bc94ac7ac02f4c76","repo":"apache/hadoop","slug":"parts-length-mismatched-d-d","errorCode":null,"errorMessage":"parts length mismatched: %d != %d","messagePattern":"parts length mismatched: (.+?) != (.+?)","errorType":"exception","errorClass":"RuntimeException","httpStatus":null,"severity":"error","filePath":"hadoop-cloud-storage-project/hadoop-tos/src/main/java/org/apache/hadoop/fs/tosfs/object/FileStore.java","lineNumber":414,"sourceCode":"    }\n\n    if (!tmpFile.renameTo(partFile)) {\n      throw new RuntimeException(\"failed to put file since rename fail.\");\n    }\n  }\n\n  @Override\n  public byte[] completeUpload(String key, String uploadId, List<Part> uploadParts) {\n    Preconditions.checkArgument(uploadParts != null && uploadParts.size() > 0,\n        \"upload parts cannot be null or empty.\");\n    File uploadDir = uploadPath(key, uploadId).toFile();\n    if (!uploadDir.exists()) {\n      throw new RuntimeException(\"cannot locate the upload id: \" + uploadId);\n    }\n\n    List<Integer> partNums = listPartNums(uploadDir);\n    if (partNums.size() != uploadParts.size()) {\n      throw new RuntimeException(String.format(\"parts length mismatched: %d != %d\",\n          partNums.size(), uploadParts.size()));\n    }\n\n    Collections.sort(partNums);\n    uploadParts.sort(Comparator.comparingInt(Part::num));\n\n    Path keyPath = path(encode(key));\n    File tmpFile = createTmpFile(keyPath.toFile());\n    try (FileOutputStream outputStream = new FileOutputStream(tmpFile);\n        FileChannel outputChannel = outputStream.getChannel()) {\n      int offset = 0;\n      for (int i = 0; i < partNums.size(); i++) {\n        Part part = uploadParts.get(i);\n        if (part.num() != partNums.get(i)) {\n          throw new RuntimeException(\n              String.format(\"part num mismatched: %d != %d\", part.num(), partNums.get(i)));\n        }\n","sourceCodeStart":396,"sourceCodeEnd":432,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-cloud-storage-project/hadoop-tos/src/main/java/org/apache/hadoop/fs/tosfs/object/FileStore.java#L396-L432","documentation":"FileStore.completeUpload() counts the part files present in the upload's staging directory and compares against the uploadParts list handed in; if the counts differ it throws 'parts length mismatched'. The caller's part list and the parts actually staged on disk no longer describe the same upload — parts are missing from the list, or the directory holds extra/stale part files.","triggerScenarios":"completeUpload where some uploadPart call never ran or failed silently, a part file was deleted from the staging dir, stale part files from an earlier numbering scheme linger, or the caller hand-built the parts list instead of using the Part objects returned by uploadPart().","commonSituations":"Loop that uploads parts but drops one on an exception path; retry frameworks that re-run a subset of parts; tests listing part files manually; concurrent writers sharing an uploadId.","solutions":["Build the parts list exclusively from the Part objects returned by uploadPart()/uploadPartCopy() and pass that exact list to completeUpload","Ensure every part upload succeeded (collect and check results) before completing","Never modify or prune files inside __STAGING__/<key>/<uploadId> by hand; if the dir is suspect, abort the upload and start a new one","Give each concurrent writer its own uploadId via createMultipartUpload instead of sharing one"],"exampleFix":"// before: hand-built parts list that lost part 1\nList<Part> parts = List.of(new Part(2, len2, etag2)); // parts length mismatched: 1 != 2\n// after: collect exactly what uploadPart returned, in upload order\nList<Part> parts = new ArrayList<>();\nfor (int n = 1; n <= totalParts; n++) {\n  parts.add(storage.uploadPart(key, uploadId, n, providerFor(n), lenFor(n)));\n}\nstorage.completeUpload(key, uploadId, parts);","handlingStrategy":"validation","validationCode":"List<Part> parts = new ArrayList<>();\nfor (int n = 1; n <= totalParts; n++) {\n  Part p = storage.uploadPart(key, uploadId, n, providerFor(n), lenFor(n));\n  if (p == null) throw new IOException(\"part \" + n + \" not uploaded\");\n  parts.add(p);\n}\nif (parts.size() != totalParts) {\n  throw new IOException(\"staged \" + parts.size() + \" parts, expected \" + totalParts);\n}\nstorage.completeUpload(key, uploadId, parts);","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Build the parts list only from Part objects returned by uploadPart/uploadPartCopy","Check every part-upload result before completing; a silently skipped part guarantees this failure","Never let tests or scripts add/remove files inside __STAGING__/<key>/<uploadId>","Isolate concurrent writers with distinct uploadIds (createMultipartUpload per writer)"],"tags":["hadoop","tos","filestore","multipart-upload","data-integrity"],"backgroundTag":"multipart-parts-mismatch","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-23T01:17:44.959Z"}