{"record":{"id":"f7d28d9b5e666de9","repo":"apache/hadoop","slug":"upload-directory-s-already-exits","errorCode":null,"errorMessage":"Upload directory %s already exits","messagePattern":"Upload directory (.+?) already exits","errorType":"exception","errorClass":"RuntimeException","httpStatus":null,"severity":"error","filePath":"hadoop-cloud-storage-project/hadoop-tos/src/main/java/org/apache/hadoop/fs/tosfs/object/FileStore.java","lineNumber":542,"sourceCode":"                      id.toFile().getName(), MIN_PART_SIZE, MAX_PART_COUNT));\n            } catch (IOException e) {\n              throw new RuntimeException(e);\n            }\n          })\n          .sorted()\n          .collect(Collectors.toList());\n    } catch (IOException e) {\n      throw new RuntimeException(e);\n    }\n  }\n\n  @Override\n  public Part uploadPartCopy(\n      String srcKey, String dstKey, String uploadId, int partNum, long copySourceRangeStart,\n      long copySourceRangeEnd) {\n    File uploadDir = uploadPath(dstKey, uploadId).toFile();\n    if (!uploadDir.exists()) {\n      throw new RuntimeException(String.format(\"Upload directory %s already exits\", uploadDir));\n    }\n    File partFile = new File(uploadDir, String.valueOf(partNum));\n    int fileSize = (int) (copySourceRangeEnd - copySourceRangeStart + 1);\n    try (InputStream is = get(srcKey, copySourceRangeStart, fileSize).stream();\n        FileOutputStream fos = new FileOutputStream(partFile)) {\n      byte[] data = new byte[fileSize];\n      IOUtils.readFully(is, data);\n      fos.write(data);\n      return new Part(partNum, fileSize, DigestUtils.md5Hex(data));\n    } catch (IOException e) {\n      throw new RuntimeException(e);\n    }\n  }\n\n  @Override\n  public void copy(String srcKey, String dstKey) {\n    Preconditions.checkArgument(!Strings.isNullOrEmpty(srcKey), \"Src key should not be empty.\");\n    File file = path(encode(srcKey)).toFile();","sourceCodeStart":524,"sourceCodeEnd":560,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-cloud-storage-project/hadoop-tos/src/main/java/org/apache/hadoop/fs/tosfs/object/FileStore.java#L524-L560","documentation":"FileStore.uploadPartCopy() throws 'Upload directory %s already exits' — note the message is inverted (and misspelled): the code checks `if (!uploadDir.exists())` and throws, so the exception actually means the staging directory for <root>/__STAGING__/<encodedDstKey>/<uploadId> is MISSING, not already present. The uploadId is unknown for this destination key, most often because the multipart upload was created for a different key string or never created at all.","triggerScenarios":"uploadPartCopy(srcKey, dstKey, uploadId, ...) where no createMultipartUpload(dstKey) with that exact uploadId has run (or it was already completed/aborted), or dstKey differs from the key the upload was created for — including cases where PrefixStorage adds/removes a prefix so the encoded path no longer matches.","commonSituations":"Copy flows that create the upload for key A but copy parts into key B; using a srcKey-derived uploadId instead of the one from createMultipartUpload(dstKey); prefix-handling wrappers passing keys with inconsistent trailing slashes or encoding; retrying after the upload was completed.","solutions":["Read the message as 'upload directory missing': always call createMultipartUpload(dstKey) first and pass its uploadId together with the exact same dstKey to uploadPartCopy","Log/compare the dstKey string at create and at copy time — they must be byte-identical (same encoding, same prefix, same trailing slash)","After completeUpload or abortMultipartUpload consumed the upload, start a new createMultipartUpload rather than reusing the id","If you wrap ObjectStorage (as PrefixStorage does), ensure the wrapper applies the same prefix transformation on every call"],"exampleFix":"// before: upload created for one key, parts copied into another\nMultipartUpload u = storage.createMultipartUpload(\"a/b.txt\");\nstorage.uploadPartCopy(\"src\", \"a/c.txt\", u.uploadId(), 1, 0, 99); // Upload directory ... already exits (means: missing)\n// after: create the upload for the exact destination key\nMultipartUpload u = storage.createMultipartUpload(dstKey);\nPart p = storage.uploadPartCopy(srcKey, dstKey, u.uploadId(), 1, 0, 99);","handlingStrategy":"validation","validationCode":"// create the upload for the exact destination key, then immediately copy parts\nMultipartUpload upload = storage.createMultipartUpload(dstKey);\nString id = upload.uploadId();\n// reuse the SAME dstKey string object/value for every call of this upload\nPart p = storage.uploadPartCopy(srcKey, dstKey, id, 1, copyStart, copyEnd);","typeGuard":null,"tryCatchPattern":"try {\n  Part p = storage.uploadPartCopy(srcKey, dstKey, uploadId, n, start, end);\n} catch (RuntimeException e) {\n  if (e.getMessage() != null && e.getMessage().contains(\"already exits\")) {\n    // message is inverted: it means the upload staging dir is MISSING for this dstKey/uploadId\n    throw new IllegalStateException(\"No multipart upload \" + uploadId\n        + \" for destination key \" + dstKey + \"; call createMultipartUpload(dstKey) first\", e);\n  }\n  throw e;\n}","preventionTips":["Always pair uploadPartCopy(dstKey, id) with createMultipartUpload(dstKey) returning that same id","Pass the identical dstKey string at create and copy time — same prefix, same encoding, same trailing slash","If you wrap ObjectStorage (prefixing/rename logic), apply the transformation uniformly in both methods","Remember the thrown message text is inverted ('already exits' == directory missing) when reading logs"],"tags":["hadoop","tos","filestore","multipart-upload","upload-part-copy"],"backgroundTag":"multipart-upload-not-found","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}