apache/beam · error · IOException

Error while attempting to create bucket gs://

Error message

Error while attempting to create bucket gs://%s for project %s

What it means

GcsUtilV1.createBucket converts an InterruptedException during the GCS bucket insert call into this IOException. A bucket-already-exists conflict is handled separately as FileAlreadyExistsException, so this error means bucket creation was interrupted rather than conflicting. The interrupt flag is restored before throwing.

Solutions

Ensure bucket creation happens before cancellation-prone phases, or pre-create the bucket out of band
Catch the IOException and check for InterruptedException cause to handle cancellation gracefully
Retry creation on a fresh, non-interrupted thread

Example fix

// before
gcsUtil.createBucket(projectId, bucket);
// after
try {
  gcsUtil.createBucket(projectId, bucket);
} catch (IOException e) {
  if (e.getCause() instanceof InterruptedException) {
    Thread.currentThread().interrupt();
    throw e; // treat as cancellation
  }
  throw e;
}

Defensive patterns

Strategy: try-catch

Validate before calling

// create only if missing
if (!gcsUtil.bucketExists(bucket)) { gcsUtil.createBucket(projectId, bucket); }

Type guard

boolean isInterruption(IOException e) { return e.getCause() instanceof InterruptedException; }

Try / catch

try {
  gcsUtil.createBucket(projectId, bucket);
} catch (FileAlreadyExistsException e) {
  // tolerate pre-existing bucket
} catch (IOException e) {
  if (e.getCause() instanceof InterruptedException) { Thread.currentThread().interrupt(); }
  throw e;
}

Prevention

Pre-create buckets via infrastructure tooling
Treat FileAlreadyExistsException as success in idempotent setup
Avoid creating buckets during shutdown

When it happens

Trigger: Calling createBucket(projectId, bucket) while the executing thread is interrupted, e.g. during pipeline cancellation or executor shutdown.

Common situations: Pipelines cancelled while creating a staging/temp bucket; test threads interrupted mid-setup.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/728372fcced2b5fe. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/extensions/google-cloud-platform-core/src/main/java/org/apache/beam/sdk/extensions/gcp/util/GcsUtilV1.java:900

                return false;
              }
              return RETRY_DETERMINER.shouldRetry(e);
            }
          },
          IOException.class,
          sleeper);
      return;
    } catch (GoogleJsonResponseException e) {
      if (errorExtractor.accessDenied(e)) {
        throw new AccessDeniedException(bucket.getName(), null, e.getMessage());
      }
      if (errorExtractor.itemAlreadyExists(e)) {
        throw new FileAlreadyExistsException(bucket.getName(), null, e.getMessage());
      }
      throw e;
    } catch (InterruptedException e) {
      Thread.currentThread().interrupt();
      throw new IOException(
          String.format(
              "Error while attempting to create bucket gs://%s for project %s",
              bucket.getName(), projectId),
          e);
    }
  }

  @VisibleForTesting
  void removeBucket(Bucket bucket, BackOff backoff, Sleeper sleeper) throws IOException {
    Storage.Buckets.Delete getBucket = storageClient.buckets().delete(bucket.getName());

    try {
      ResilientOperation.retry(
          getBucket::execute,
          backoff,
          new RetryDeterminer<IOException>() {
            @Override
            public boolean shouldRetry(IOException e) {

View on GitHub (pinned to 12126d8942)