{"record":{"id":"ca2db2751351a3d8","repo":"apache/hadoop","slug":"multi-part-upload-with-id-uploadid-to-key","errorCode":null,"errorMessage":"Multi-part upload with id '{uploadId}' to {key}","messagePattern":"Multi-part upload with id '(.+?)' to (.+?)","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"critical","filePath":"hadoop-tools/hadoop-aliyun/src/main/java/org/apache/hadoop/fs/aliyun/oss/AliyunOSSBlockOutputStream.java","lineNumber":273,"sourceCode":"  private List<PartETag> waitForAllPartUploads() throws IOException {\n    LOG.debug(\"Waiting for {} uploads to complete\", partETagsFutures.size());\n    try {\n      return Futures.allAsList(partETagsFutures).get();\n    } catch (InterruptedException ie) {\n      LOG.warn(\"Interrupted partUpload\", ie);\n      Thread.currentThread().interrupt();\n      return null;\n    } catch (ExecutionException ee) {\n      //there is no way of recovering so abort\n      //cancel all partUploads\n      LOG.debug(\"While waiting for upload completion\", ee);\n      LOG.debug(\"Cancelling futures\");\n      for (ListenableFuture<PartETag> future : partETagsFutures) {\n        future.cancel(true);\n      }\n      //abort multipartupload\n      store.abortMultipartUpload(key, uploadId);\n      throw new IOException(\"Multi-part upload with id '\" + uploadId\n        + \"' to \" + key, ee);\n    }\n  }\n}\n","sourceCodeStart":255,"sourceCodeEnd":278,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-tools/hadoop-aliyun/src/main/java/org/apache/hadoop/fs/aliyun/oss/AliyunOSSBlockOutputStream.java#L255-L278","documentation":"Thrown by AliyunOSSBlockOutputStream.waitForAllPartUploads() when one of the ListenableFuture<PartETag> part uploads failed with an ExecutionException. On this path the code cancels all remaining part futures, calls store.abortMultipartUpload(key, uploadId) so OSS discards the incomplete parts, and wraps the cause in an IOException carrying the uploadId and destination key.","triggerScenarios":"Any individual part PUT to OSS failing: 403 signature/permission errors, expired STS token mid-upload, network reset, bucket-level throttling (RequestTimeTooSkewed, SlowDown), or the OSS client throwing inside the executor. The exception's cause holds the actual OSS error.","commonSituations":"STS security token expiring during a long write of a large file; transient network issues between the cluster and the OSS endpoint; RAM policy missing oss:PutObject on the bucket; throttling under heavy parallel part uploads (fs.oss.multipart.size too small, many concurrent tasks).","solutions":["Inspect the nested cause of the IOException — it contains the real OSS SDK error (e.g., InvalidAccessKeyId, RequestTimeTooSkewed, SocketTimeout) and fix that","For long writes with temporary credentials, refresh STS tokens with a longer validity or use a credentials provider that auto-refreshes","Retry the write at the job/task level; the upload was aborted so no partial object remains","For throttling, increase fs.oss.multipart.size so fewer part requests are issued, or reduce concurrent writers"],"exampleFix":null,"handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"catch (IOException e) { Throwable cause = e.getCause(); if (cause instanceof OSSException) { String code = ((OSSException) cause).getErrorCode(); /* handle InvalidAccessKeyId / RequestTimeTooSkewed / SlowDown */ } throw e; }","preventionTips":["Use STS tokens with validity longer than the maximum expected write duration, or auto-refreshing providers","Retry the write at the job/task level — the connector already aborted the multipart upload","Inspect the nested cause before changing config; it carries the real OSS error code"],"tags":["aliyun-oss","multipart-upload","upload-failure","hadoop-connector"],"backgroundTag":"multipart-upload-failure","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-23T01:17:44.959Z"}