{"record":{"id":"dd43f023bb09c64a","repo":"apache/hadoop","slug":"failed-to-multipart-upload-to-oss-abort-it","errorCode":null,"errorMessage":"Failed to multipart upload to oss, abort it.","messagePattern":"Failed to multipart upload to oss, abort it\\.","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"critical","filePath":"hadoop-tools/hadoop-aliyun/src/main/java/org/apache/hadoop/fs/aliyun/oss/AliyunOSSBlockOutputStream.java","lineNumber":152,"sourceCode":"          // zero size file\n          store.storeEmptyFile(key);\n        } else {\n          OSSDataBlocks.BlockUploadData uploadData = dataBlock.startUpload();\n          if (uploadData.hasFile()) {\n            store.uploadObject(key, uploadData.getFile());\n          } else {\n            store.uploadObject(key,\n                uploadData.getUploadStream(), dataBlock.dataSize());\n          }\n        }\n      } else {\n        if (blockWritten > 0) {\n          uploadCurrentBlock();\n        }\n        // wait for the partial uploads to finish\n        final List<PartETag> partETags = waitForAllPartUploads();\n        if (null == partETags) {\n          throw new IOException(\"Failed to multipart upload to oss, abort it.\");\n        }\n        store.completeMultipartUpload(key, uploadId,\n            new ArrayList<>(partETags));\n      }\n    } finally {\n      cleanupWithLogger(LOG, getActiveBlock(), blockFactory);\n      closed.set(true);\n    }\n  }\n\n  @Override\n  public synchronized void write(int b) throws IOException {\n    singleByte[0] = (byte)b;\n    write(singleByte, 0, 1);\n  }\n\n  @Override\n  public synchronized void write(byte[] b, int off, int len)","sourceCodeStart":134,"sourceCodeEnd":170,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-tools/hadoop-aliyun/src/main/java/org/apache/hadoop/fs/aliyun/oss/AliyunOSSBlockOutputStream.java#L134-L170","documentation":"Thrown at the end of AliyunOSSBlockOutputStream.close() during a multipart upload. waitForAllPartUploads() returns null only when the waiting thread was interrupted (the catch of InterruptedException re-interrupts and returns null), so this IOException means the multipart upload could not be completed because part-upload futures were abandoned, and the object was not assembled.","triggerScenarios":"The writer thread is interrupted (task kill, executor shutdownNow) while close() is waiting for part uploads of the final partial block; JVM shutdown hooks interrupt worker threads during commit; the bounded thread pool backing uploads was shut down before close() finished.","commonSituations":"Spark/MapReduce task cancellation racing the job committer's commitTask; executor shutdown order bugs where fs.oss.buffer.dirs upload threads are terminated first; long close() latency combined with aggressive client-side timeouts that interrupt threads.","solutions":["Check for and log Thread.interrupted status and task-kill signals; if the task was deliberately killed, expect this failure and let the framework's task retry produce the output","Avoid shutting down the shared ExecutorService used for uploads before all output streams are closed","Retry the failing job/stage: since the multipart upload never completed, the destination object is absent or partial, so a clean re-run is safe","If interrupts are unexpected, look for over-aggressive watchdogs or future.cancel(true) in your code touching the upload futures"],"exampleFix":null,"handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"catch (IOException e) { // interrupted multipart close: rely on task-level retry if (Thread.currentThread().isInterrupted()) { Thread.currentThread().interrupt(); } throw e; }","preventionTips":["Never shut down the shared upload ExecutorService before all output streams are closed","Expect and allow framework task retry when tasks are killed during commit","Keep close() on the writer thread, not a racing shutdown hook"],"tags":["aliyun-oss","multipart-upload","interrupt","hadoop-connector"],"backgroundTag":"multipart-upload-aborted","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-23T01:17:44.959Z"}