{"record":{"id":"839f21d08d47bdfb","repo":"apache/hadoop","slug":"multipart-upload-failed","errorCode":null,"errorMessage":"Multipart upload failed","messagePattern":"Multipart upload failed","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"critical","filePath":"hadoop-cloud-storage-project/hadoop-bos/src/main/java/org/apache/hadoop/fs/bos/BosOutputStream.java","lineNumber":404,"sourceCode":"        int index = 0;\n        for (Future future : this.futureList) {\n          future.get();\n          index += 1;\n          LOG.debug(\n              \"future.get() index: {} is done\",\n              index);\n        }\n      } catch (Exception e) {\n        uploadException = e;\n        LOG.warn(\n            \"catch exception when waiting\"\n                + \" UploadPartThread done: \",\n            e);\n      }\n\n      if (uploadException != null) {\n        abortMultipartUpload();\n        throw new IOException(\n            \"Multipart upload failed\", uploadException);\n      }\n\n      LOG.debug(\n          \"success to wait upload part threads done\");\n\n      LOG.debug(\n          \"Size of eTags is {}. blkIndex is {}\",\n          this.eTags.size(), this.blkIndex);\n\n      if (this.eTags.size() != this.blkIndex - 1) {\n        abortMultipartUpload();\n        throw new IOException(\n            \"Multipart upload incomplete: expected \"\n                + (this.blkIndex - 1) + \" parts but got \"\n                + this.eTags.size());\n      }\n","sourceCodeStart":386,"sourceCodeEnd":422,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-cloud-storage-project/hadoop-bos/src/main/java/org/apache/hadoop/fs/bos/BosOutputStream.java#L386-L422","documentation":"At close(), BosOutputStream joins all UploadPartThreads, collecting any failure into uploadException; if one exists it calls abortMultipartUpload() — deleting every uploaded part server-side — and throws IOException('Multipart upload failed') with the part exception as cause. The object is never created; all written data is discarded.","triggerScenarios":"One or more part-upload threads fail during the final flush/close: network failure on the last parts, 403 from expired credentials, bucket deleted or ACL changed mid-write, sustained throttling.","commonSituations":"Job fails at commit stage after a long upload because the token expired before close; BOS maintenance window overlapping the write; bucket permissions changed while the job ran.","solutions":["Inspect getCause() of the IOException — it is the actual part-upload failure and dictates the fix","Re-run the write from the beginning: abort already removed partial parts and no object exists; the stream is not resumable","Make the producing job/checkpoint idempotent so the retry is safe at file granularity","For very large files, use credentials/STS TTLs longer than the worst-case write duration"],"exampleFix":null,"handlingStrategy":"retry","validationCode":null,"typeGuard":"static boolean isMultipartUploadFailed(IOException e) {\n  return \"Multipart upload failed\".equals(e.getMessage());\n}","tryCatchPattern":"catch (IOException e) {\n  if (isMultipartUploadFailed(e) && e.getCause() != null) {\n    LOG.error(\"root cause\", e.getCause());\n    // abort already ran server-side; safe to recreate the object from scratch\n    rewriteFromSource(path);\n  } else { throw e; }\n}","preventionTips":["Design commit-time retries: the abort leaves no partial object, so whole-file retry is safe","Use STS/credential TTL larger than worst-case upload duration","Alert on close()-phase failures — they discard the entire file"],"tags":["bos","multipart-upload","upload-aborted","data-loss","hadoop"],"backgroundTag":"multipart-upload-failed","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}