{"record":{"id":"26a7137f6576e8db","repo":"apache/hadoop","slug":"failed-to-delete-temporary-files-while-closing-str","errorCode":null,"errorMessage":"Failed to delete temporary files while closing stream: '%s'","messagePattern":"Failed to delete temporary files while closing stream: '(.+?)'","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-cloud-storage-project/hadoop-gcp/src/main/java/org/apache/hadoop/fs/gs/GoogleHadoopOutputStream.java","lineNumber":267,"sourceCode":"    commitTempFile();\n\n    try {\n      tmpOut.close();\n    } finally {\n      tmpOut = null;\n    }\n    tmpGcsPath = null;\n    tmpIndex = -1;\n\n    LOG.trace(\"close(): Awaiting {} deletionFutures\", tmpDeletionFutures.size());\n    for (Future<?> deletion : tmpDeletionFutures) {\n      try {\n        deletion.get();\n      } catch (ExecutionException | InterruptedException e) {\n        if (e instanceof InterruptedException) {\n          Thread.currentThread().interrupt();\n        }\n        throw new IOException(\n            String.format(\n                \"Failed to delete temporary files while closing stream: '%s'\", dstGcsPath),\n            e);\n      }\n    }\n  }\n\n  private void throwIfNotOpen() throws IOException {\n    if (tmpOut == null) {\n      throw new ClosedChannelException();\n    }\n  }\n\n  @Override\n  public boolean hasCapability(String capability) {\n    checkArgument(!isNullOrEmpty(capability), \"capability must not be null or empty string\");\n    switch (Ascii.toLowerCase(capability)) {\n    case StreamCapabilities.HFLUSH:","sourceCodeStart":249,"sourceCodeEnd":285,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-cloud-storage-project/hadoop-gcp/src/main/java/org/apache/hadoop/fs/gs/GoogleHadoopOutputStream.java#L249-L285","documentation":"GoogleHadoopOutputStream buffers large writes into temporary GCS objects and deletes them asynchronously; close() waits on those deletion futures (tmpDeletionFutures). If any future fails (ExecutionException) or the closing thread is interrupted, close() throws IOException wrapping the cause. Note the data in the destination object is already committed - only temp cleanup failed.","triggerScenarios":"out.close() while a background deletion fails: GCS API/network errors, service account lacking storage.objects.delete on the temp location, or executor shutdown; thread interruption during close (the code restores the interrupt flag before throwing).","commonSituations":"Aggressive shutdown hooks interrupting writer threads; credentials expiring or permissions changed during very long writes; leftover gs://<bucket>/<tmp-dir> objects accumulating after this failure; executors shared with code that shuts them down early.","solutions":["Inspect the wrapped cause (e.getCause()) to identify the real failure - permission, network, or interrupt.","Verify the service account can delete objects under the configured temporary directory; clean stale temp objects (hadoop- prefixed leftovers) after failures.","Avoid interrupting writer threads during close; ensure the connector's executor is not externally shut down.","Treat this as a cleanup warning for the committed output: data is written, but investigate why deletion failed."],"exampleFix":"// before\ntry (FSDataOutputStream out = fs.create(path)) {\n  out.write(data);\n} // close() may throw 'Failed to delete temporary files while closing stream'\n\n// after\ntry (FSDataOutputStream out = fs.create(path)) {\n  out.write(data);\n} catch (IOException e) {\n  // output is committed; diagnose cleanup failure via e.getCause()\n  LOG.warn(\"temp cleanup failed for {}\", path, e.getCause());\n  // optionally sweep the configured temp dir for leftovers\n}","handlingStrategy":"try-catch","validationCode":"null","typeGuard":null,"tryCatchPattern":"try (FSDataOutputStream out = fs.create(path)) {\n  out.write(data);\n} catch (IOException e) {\n  if (e.getMessage() != null && e.getMessage().contains(\"Failed to delete temporary files\")) {\n    // committed output is intact; inspect e.getCause() (perm/network/interrupt) and sweep leftover temp objects\n  } else {\n    throw e;\n  }\n}","preventionTips":["Grant the writer identity storage.objects.delete on the configured temp location.","Never interrupt writer threads during close(); let shutdown hooks drain gracefully.","After this failure, sweep the connector's temp directory for stale objects to avoid unbounded accumulation."],"tags":["gcs","hadoop","output-stream","close","cleanup","interrupt"],"backgroundTag":"temp-file-cleanup-failure","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}