apache/beam · error · FileNotFoundException

The specified bucket does not exist: gs://%s

Error message

The specified bucket does not exist: gs://%s

What it means

GcsUtilV2.getBucket fetches a Bucket via the Storage client and throws FileNotFoundException when Storage.get(bucketName) returns null, meaning the bucket does not exist (or is invisible to the credentials). StorageExceptions are translated separately.

Source

Thrown at sdks/java/extensions/google-cloud-platform-core/src/main/java/org/apache/beam/sdk/extensions/gcp/util/GcsUtilV2.java:440

    rewriteHelper(srcPaths, dstPaths, false, MissingStrategy.FAIL_IF_MISSING, strategy);
  }

  public void move(
      Iterable<GcsPath> srcPaths,
      Iterable<GcsPath> dstPaths,
      MissingStrategy srcMissing,
      OverwriteStrategy dstOverwrite)
      throws IOException {
    rewriteHelper(srcPaths, dstPaths, true, srcMissing, dstOverwrite);
  }

  /** Get the {@link Bucket} from Cloud Storage path or propagates an exception. */
  public Bucket getBucket(GcsPath path, BucketGetOption... options) throws IOException {
    String bucketName = path.getBucket();
    try {
      Bucket bucket = storage.get(bucketName, options);
      if (bucket == null) {
        throw new FileNotFoundException(
            String.format("The specified bucket does not exist: gs://%s", bucketName));
      }
      return bucket;
    } catch (StorageException e) {
      throw translateStorageException(bucketName, null, e);
    }
  }

  /** Returns whether the GCS bucket exists and is accessible. */
  public boolean bucketAccessible(GcsPath path) {
    try {
      // Fetch only the name field to minimize data transfer
      getBucket(path, BucketGetOption.fields(BucketField.NAME));
      return true;
    } catch (IOException e) {
      return false;
    }
  }

View on GitHub (pinned to 12126d8942)

Solutions

1. Verify the bucket: gsutil ls gs://<bucket> or check the Cloud Console.
2. Confirm the bucket name in pipeline options (tempLocation, stagingLocation) is exact and lowercase.
3. Ensure the credential's project can access the bucket; grant roles/storage.admin or objectViewer on it.
4. Create the bucket if it was never provisioned (gsutil mb).

Example fix

// before
options.setTempLocation("gs://my-temp-bucket/path"); // typo'd bucket
// after: validate early
new GcsOptionsValidator()/* or */; // gcloud storage buckets describe gs://my-temp-bucket
options.setTempLocation("gs://correct-temp-bucket/path");

// fail fast at job setup
String bucket = GcsPath.fromUri(options.getTempLocation()).getBucket();
if (!gcsUtil.bucketAccessible(bucket)) {
  throw new IllegalArgumentException("Bucket missing or inaccessible: " + bucket);
}

try {
  Bucket b = gcsUtil.getBucket(path);
} catch (FileNotFoundException e) {
  // create the bucket or correct the configured location before proceeding
}

When it happens

Trigger: Any call path through getBucket — bucketAccessible, verifyBucketAccessible, or bucket(path) — where the bucket name doesn't exist or the caller has no permission to see it (a 403 can surface as null in some configurations).

Common situations: Typo in the --gcsEndpoint / staging bucket / tempLocation bucket name; using a bucket in a different project than the credential; bucket deleted or renamed between job submission and execution; regional bucket name mismatches.

Understand the failure class

Background: 'Could not be found', 'does not exist', 'not found in database': the resource-not-found family when an ID, slug, key, or URI lookup comes back empty — this error's family across 20 libraries.

Related errors

• Unable to find schema for ${identifier}SchemaTransformProvid
• Error completing file copies with retries, sample: from %s t
• Interrupted backoff of file copies with retries, sample: fro
• Skipping dest existence is only supported within a bucket.
• Error trying to delete %s: %s

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/a7e46b97bc3096aa. Report an issue: GitHub.