{"record":{"id":"b360b9e46bfb3d23","repo":"apache/beam","slug":"interrupted-while-executing-batch-gcs-request","errorCode":null,"errorMessage":"Interrupted while executing batch GCS request","messagePattern":"Interrupted while executing batch GCS request","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"sdks/java/extensions/google-cloud-platform-core/src/main/java/org/apache/beam/sdk/extensions/gcp/util/GcsUtilV1.java","lineNumber":978,"sourceCode":"      } catch (ExecutionException e) {\n        if (e.getCause() instanceof FileNotFoundException) {\n          throw (FileNotFoundException) e.getCause();\n        }\n        throw new IOException(\"Error executing batch GCS request\", e);\n      } finally {\n        // Give the other batches a chance to complete in error cases.\n        executor.shutdown();\n        if (!executor.awaitTermination(5, TimeUnit.MINUTES)) {\n          LOG.warn(\"Taking over 5 minutes to flush gcs op batches after error\");\n          executor.shutdownNow();\n          if (!executor.awaitTermination(5, TimeUnit.MINUTES)) {\n            LOG.warn(\"Took over 10 minutes to flush gcs op batches after error and interruption.\");\n          }\n        }\n      }\n    } catch (InterruptedException e) {\n      Thread.currentThread().interrupt();\n      throw new IOException(\"Interrupted while executing batch GCS request\", e);\n    }\n  }\n\n  /**\n   * Makes get {@link BatchInterface BatchInterfaces}.\n   *\n   * @param paths {@link GcsPath GcsPaths}.\n   * @param results mutable {@link List} for return values.\n   * @return {@link BatchInterface BatchInterfaces} to execute.\n   * @throws IOException\n   */\n  @VisibleForTesting\n  List<BatchInterface> makeGetBatches(\n      Collection<GcsPath> paths, List<StorageObjectOrIOException[]> results) throws IOException {\n    List<BatchInterface> batches = new ArrayList<>();\n    for (List<GcsPath> filesToGet :\n        Lists.partition(Lists.newArrayList(paths), MAX_REQUESTS_PER_BATCH)) {\n      BatchInterface batch = batchRequestSupplier.get();","sourceCodeStart":960,"sourceCodeEnd":996,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/extensions/google-cloud-platform-core/src/main/java/org/apache/beam/sdk/extensions/gcp/util/GcsUtilV1.java#L960-L996","documentation":"GcsUtilV1.executeBatches catches InterruptedException while waiting for batched GCS futures to complete and rethrows it as this IOException after restoring the interrupt flag. It means the batch flush was interrupted before all operations completed, leaving some operations possibly unexecuted.","triggerScenarios":"Calling any batched GCS operation while the awaiting thread is interrupted — pipeline cancellation, executor shutdown, or test timeouts interrupting workers.","commonSituations":"Cancelling a pipeline mid-flush of GCS batch operations; CI timeouts interrupting test threads during cleanup.","solutions":["Allow batch flushes to complete before interrupting/cancelling","Catch IOException, check for InterruptedException cause, and treat as cancellation","Re-issue the batch if idempotent (e.g. deletes) since completion state is uncertain"],"exampleFix":"// before\ngcsUtil.removeBatches(batches); // may throw this on interrupt\n// after\ntry {\n  gcsUtil.removeBatches(batches);\n} catch (IOException e) {\n  if (e.getCause() instanceof InterruptedException) {\n    Thread.currentThread().interrupt();\n    throw e; // cancellation path\n  }\n  throw e;\n}","handlingStrategy":"try-catch","validationCode":null,"typeGuard":"boolean isInterruption(IOException e) { return e.getCause() instanceof InterruptedException; }","tryCatchPattern":"try {\n  gcsUtil.removeBatches(batches);\n} catch (IOException e) {\n  if (e.getCause() instanceof InterruptedException) {\n    Thread.currentThread().interrupt();\n    // treat as cancellation, maybe re-issue batch if idempotent\n  }\n  throw e;\n}","preventionTips":["Flush batches before shutdown begins","Re-issue idempotent batches after interruption","Avoid interrupting threads mid-flush"],"tags":["gcs","interrupted","concurrency"],"backgroundTag":"thread-interrupted","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}