{"record":{"id":"fb2a821bfe654598","repo":"apache/hadoop","slug":"cannot-locate-the-upload-id-s","errorCode":null,"errorMessage":"cannot locate the upload id: %s","messagePattern":"cannot locate the upload id: (.+?)","errorType":"exception","errorClass":"RuntimeException","httpStatus":null,"severity":"error","filePath":"hadoop-cloud-storage-project/hadoop-tos/src/main/java/org/apache/hadoop/fs/tosfs/object/FileStore.java","lineNumber":351,"sourceCode":"      return new MultipartUpload(key, uploadId, MIN_PART_SIZE, MAX_PART_COUNT);\n    } else {\n      throw new RuntimeException(\"Failed to create MultipartUpload with key: \" + key);\n    }\n  }\n\n  private Path uploadPath(String key, String uploadId) {\n    return Paths.get(root, STAGING_DIR, encode(key), uploadId);\n  }\n\n  @Override\n  public Part uploadPart(\n      String key, String uploadId, int partNum,\n      InputStreamProvider streamProvider, long contentLength) {\n    Preconditions.checkArgument(!Strings.isNullOrEmpty(key), \"Key should not be empty.\");\n\n    File uploadDir = uploadPath(key, uploadId).toFile();\n    if (!uploadDir.exists()) {\n      throw new RuntimeException(\"cannot locate the upload id: \" + uploadId);\n    }\n\n    File partFile = new File(uploadDir, String.valueOf(partNum));\n    copyInputStreamToFile(streamProvider.newStream(), partFile, contentLength);\n\n    try {\n      byte[] data = Files.readAllBytes(partFile.toPath());\n      return new Part(partNum, data.length, DigestUtils.md5Hex(data));\n    } catch (IOException e) {\n      LOG.error(\"failed to locate the part file: {}\", partFile.getAbsolutePath());\n      throw new RuntimeException(e);\n    }\n  }\n\n  private static void appendInputStreamToFile(InputStream in, File partFile, long contentLength) {\n    try (FileOutputStream out = new FileOutputStream(partFile, true)) {\n      long copiedBytes = IOUtils.copyLarge(in, out, 0, contentLength);\n","sourceCodeStart":333,"sourceCodeEnd":369,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-cloud-storage-project/hadoop-tos/src/main/java/org/apache/hadoop/fs/tosfs/object/FileStore.java#L333-L369","documentation":"FileStore.uploadPart() resolves the upload's staging directory <root>/__STAGING__/<encodedKey>/<uploadId> and throws 'cannot locate the upload id' when that directory does not exist. In this local emulation the uploadId is only valid as long as its staging directory exists — once completeUpload or abortMultipartUpload deletes it (or the root changes), the id is dead.","triggerScenarios":"uploadPart(key, uploadId, partNum, ...) with an uploadId that was never created by createMultipartUpload in this store, one whose upload was already completed or aborted (staging dir removed), or when the process resolves a different fs.filestore.endpoint/FILE_STORAGE_ROOT than the one that created the upload.","commonSituations":"Retrying a part upload after the upload was already completed or aborted; tests that create the FileStore per test method but share uploadIds across tests; cleanup (@After) deleting the staging tree while async part writers are still running; hand-copied uploadId strings.","solutions":["Only use uploadId values returned by createMultipartUpload() on the same ObjectStorage instance/root, within that upload's lifetime","After a completeUpload or abortMultipartUpload, always start a fresh createMultipartUpload instead of reusing the old id","Make sure every process/test resolving the store uses the same root configuration","Guard async part writers against cleanup races: shut them down before deleting the staging/root directories"],"exampleFix":"// before: stale id from an earlier (already completed) upload\nstorage.uploadPart(key, oldUploadId, 1, provider, len); // RuntimeException: cannot locate the upload id\n// after: create and use the upload in one lifecycle\nMultipartUpload upload = storage.createMultipartUpload(key);\nPart part = storage.uploadPart(key, upload.uploadId(), 1, provider, len);\nstorage.completeUpload(key, upload.uploadId(), List.of(part));","handlingStrategy":"validation","validationCode":"// track the upload lifecycle in the calling code\njava.util.Set<String> activeUploads = java.util.concurrent.ConcurrentHashMap.newKeySet();\nMultipartUpload upload = storage.createMultipartUpload(key);\nactiveUploads.add(upload.uploadId());\n\n// later, before uploading a part:\nif (!activeUploads.contains(uploadId)) {\n  throw new IllegalStateException(\"uploadId not active (completed/aborted?): \" + uploadId);\n}\nstorage.uploadPart(key, uploadId, partNum, provider, len);","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Use uploadIds only within the lifetime of their createMultipartUpload -> complete/abort window","Remove the id from your active set as soon as completeUpload or abortMultipartUpload returns","Never copy uploadIds between tests, processes, or differently-configured stores","Shut down async part writers before any cleanup that deletes the staging tree"],"tags":["hadoop","tos","filestore","multipart-upload","staging"],"backgroundTag":"multipart-upload-not-found","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}