apache/beam · error · IOException

Interrupted while executing batch GCS request

Error message

Interrupted while executing batch GCS request

What it means

GcsUtilV1.executeBatches catches InterruptedException while waiting for batched GCS futures to complete and rethrows it as this IOException after restoring the interrupt flag. It means the batch flush was interrupted before all operations completed, leaving some operations possibly unexecuted.

Solutions

1. Allow batch flushes to complete before interrupting/cancelling
2. Catch IOException, check for InterruptedException cause, and treat as cancellation
3. Re-issue the batch if idempotent (e.g. deletes) since completion state is uncertain

Example fix

// before
gcsUtil.removeBatches(batches); // may throw this on interrupt
// after
try {
  gcsUtil.removeBatches(batches);
} catch (IOException e) {
  if (e.getCause() instanceof InterruptedException) {
    Thread.currentThread().interrupt();
    throw e; // cancellation path
  }
  throw e;
}

boolean isInterruption(IOException e) { return e.getCause() instanceof InterruptedException; }

try {
  gcsUtil.removeBatches(batches);
} catch (IOException e) {
  if (e.getCause() instanceof InterruptedException) {
    Thread.currentThread().interrupt();
    // treat as cancellation, maybe re-issue batch if idempotent
  }
  throw e;
}

When it happens

Trigger: Calling any batched GCS operation while the awaiting thread is interrupted — pipeline cancellation, executor shutdown, or test timeouts interrupting workers.

Common situations: Cancelling a pipeline mid-flush of GCS batch operations; CI timeouts interrupting test threads during cleanup.

Related errors

• Error while attempting to create bucket gs://
• Error while attempting to remove bucket gs://
• Error while attempting to verify existence of bucket gs://
• Interrupted backoff of file copies with retries, sample…
• Already bound to %r

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/b360b9e46bfb3d23. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/extensions/google-cloud-platform-core/src/main/java/org/apache/beam/sdk/extensions/gcp/util/GcsUtilV1.java:978

      } catch (ExecutionException e) {
        if (e.getCause() instanceof FileNotFoundException) {
          throw (FileNotFoundException) e.getCause();
        }
        throw new IOException("Error executing batch GCS request", e);
      } finally {
        // Give the other batches a chance to complete in error cases.
        executor.shutdown();
        if (!executor.awaitTermination(5, TimeUnit.MINUTES)) {
          LOG.warn("Taking over 5 minutes to flush gcs op batches after error");
          executor.shutdownNow();
          if (!executor.awaitTermination(5, TimeUnit.MINUTES)) {
            LOG.warn("Took over 10 minutes to flush gcs op batches after error and interruption.");
          }
        }
      }
    } catch (InterruptedException e) {
      Thread.currentThread().interrupt();
      throw new IOException("Interrupted while executing batch GCS request", e);
    }
  }

  /**
   * Makes get {@link BatchInterface BatchInterfaces}.
   *
   * @param paths {@link GcsPath GcsPaths}.
   * @param results mutable {@link List} for return values.
   * @return {@link BatchInterface BatchInterfaces} to execute.
   * @throws IOException
   */
  @VisibleForTesting
  List<BatchInterface> makeGetBatches(
      Collection<GcsPath> paths, List<StorageObjectOrIOException[]> results) throws IOException {
    List<BatchInterface> batches = new ArrayList<>();
    for (List<GcsPath> filesToGet :
        Lists.partition(Lists.newArrayList(paths), MAX_REQUESTS_PER_BATCH)) {
      BatchInterface batch = batchRequestSupplier.get();

View on GitHub (pinned to 12126d8942)