apache/beam · error · RuntimeException

Unable create default bucket.

Error message

Unable create default bucket.

What it means

GcpOptions.tryCreateDefaultBucketWithPrefix calls GcsUtil.createBucket; any IOException other than FileAlreadyExistsException is rethrown as RuntimeException 'Unable create default bucket.' Bucket creation for the auto-generated default temp bucket failed.

Source

Thrown at sdks/java/extensions/google-cloud-platform-core/src/main/java/org/apache/beam/sdk/extensions/gcp/options/GcpOptions.java:493

      LOG.info("No tempLocation specified, attempting to use default bucket: {}", bucketName);

      // Disable soft delete policy for a bucket.
      // Reference: https://cloud.google.com/storage/docs/soft-delete
      SoftDeletePolicy softDeletePolicy = new SoftDeletePolicy().setRetentionDurationSeconds(0L);

      Bucket bucket =
          new Bucket()
              .setName(bucketName)
              .setLocation(region)
              .setSoftDeletePolicy(softDeletePolicy);
      // Always try to create the bucket before checking access, so that we do not
      // race with other pipelines that may be attempting to do the same thing.
      try {
        gcsOptions.getGcsUtil().createBucket(gcsOptions.getProject(), bucket);
      } catch (FileAlreadyExistsException e) {
        LOG.debug("Bucket '{}'' already exists, verifying access.", bucketName);
      } catch (IOException e) {
        throw new RuntimeException("Unable create default bucket.", e);
      }

      // Once the bucket is expected to exist, verify that it is correctly owned
      // by the project executing the job.
      try {
        long owner = gcsOptions.getGcsUtil().bucketOwner(GcsPath.fromComponents(bucketName, ""));
        checkArgument(
            owner == projectNumber,
            "Bucket owner does not match the project from --project:" + " %s vs. %s",
            owner,
            projectNumber);
      } catch (IOException e) {
        throw new RuntimeException(
            "Unable to determine the owner of the default bucket at gs://" + bucketName, e);
      }
      return "gs://" + bucketName + "/temp/";
    }

View on GitHub (pinned to 12126d8942)

Solutions

1. Pre-create a bucket and pass it via --tempLocation so auto-creation is skipped
2. Grant roles/storage.admin (or storage.buckets.create) to the pipeline identity
3. Check bucket quota and naming rules for the generated name
4. Verify storage component is enabled and reachable

Example fix

// before
// relying on auto-created default bucket
// after
gsutil mb gs://my-project-beam-bucket
--tempLocation=gs://my-project-beam-bucket/temp/

// ensure bucket creation is permitted or skip auto-create
boolean canCreate = gcpOptions.getProject() != null; // plus storage.buckets.create IAM
String explicit = options.getTempLocation();
if (explicit != null && explicit.startsWith("gs://")) { /* auto-create will be skipped */ }

try {
  String b = gcpOptions.getDefaultBucketName();
} catch (RuntimeException e) {
  if (e.getMessage() != null && e.getMessage().contains("Unable create default bucket")) {
    gcpOptions.setTempLocation("gs://precreated-bucket/temp/");
  } else throw e;
}

When it happens

Trigger: getDefaultBucketName() infers no suitable bucket and attempts to create gs://<project>-<hash>; createBucket throws IOException due to invalid bucket name, storage.buckets.create permission denied, quota exceeded, or API/network failure.

Common situations: Service account lacking storage.buckets.create on the project; project at bucket quota; bucket name conflicts or naming constraints; GCS API disabled.

Understand the failure class

Background: "failed to write file", "Could not save figure", "Error saving remote file" — file write failed: causes and fixes across languages and libraries — this error's family across 38 libraries.

Related errors

• Unable to determine the owner of the default bucket at gs://
• Error constructing default value for gcpTempLocation: tempLo
• Unable to get project number
• The key '%s' in GCS custom audit entries exceeds the %d-char
• The value '%s' in GCS custom audit entries exceeds the %d-ch

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/75853b2e5a8e611e. Report an issue: GitHub.