apache/beam · error · RuntimeException

Could not find file %s

Error message

Could not find file %s

What it means

GcsPathValidator.verifyPathIsAccessible calls GcsUtil.verifyBucketAccessible and wraps IOException in RuntimeException formatted with the caller's errorMessage (e.g. 'Could not find file %s'). It means the GCS bucket backing the given path could not be confirmed accessible during pipeline validation.

Source

Thrown at sdks/java/extensions/google-cloud-platform-core/src/main/java/org/apache/beam/sdk/extensions/gcp/storage/GcsPathValidator.java:85

    GcsPath gcsPath = getGcsPath(path);
    checkArgument(gcsPath.isAbsolute(), "Must provide absolute paths for Dataflow");
    checkArgument(
        !gcsPath.getObject().isEmpty(),
        "Missing object or bucket in path: '%s', did you mean: 'gs://some-bucket/%s'?",
        gcsPath,
        gcsPath.getBucket());
    checkArgument(
        !gcsPath.getObject().contains("//"),
        "Dataflow Service does not allow objects with consecutive slashes");
    return gcsPath.toResourceName();
  }

  private void verifyPathIsAccessible(String path, String errorMessage) {
    GcsPath gcsPath = getGcsPath(path);
    try {
      gcpOptions.getGcsUtil().verifyBucketAccessible(gcsPath);
    } catch (IOException e) {
      throw new RuntimeException(String.format(errorMessage, path), e);
    }
  }

  private GcsPath getGcsPath(String path) {
    try {
      return GcsPath.fromUri(path);
    } catch (IllegalArgumentException e) {
      throw new IllegalArgumentException(
          String.format("Expected a valid 'gs://' path but was given '%s'", path), e);
    }
  }
}

View on GitHub (pinned to 12126d8942)

Solutions

1. Verify the path with `gsutil ls gs://bucket/path` using the same credentials
2. Correct typos and confirm the input files exist before launching
3. Grant the identity read access (roles/storage.objectViewer) on the bucket
4. Retry on transient errors; check GCS status if failures persist

Example fix

// before
--input=gs://my-bukket/data/*.json // typo
// after
--input=gs://my-bucket/data/*.json // verified via gsutil ls

// verify GCS input exists before launching the pipeline
// gsutil -q stat gs://bucket/path  (exit code 0 means accessible)
Process p = new ProcessBuilder("gsutil", "-q", "stat", gcsPath).redirectErrorStream(true).start();
if (p.waitFor() != 0) throw new IllegalArgumentException("GCS path not accessible: " + gcsPath);

boolean isGcsPath(String s) { return s != null && s.startsWith("gs://") && s.length() > 5; }

try {
  options.as(GcsOptions.class).getPathValidator().validateInputFilePatternSupported(path);
} catch (RuntimeException e) {
  if (e.getMessage() != null && e.getMessage().contains("Could not find file")) {
    throw new IllegalArgumentException("Check bucket name, object existence, and read permissions for " + path, e);
  }
  throw e;
}

When it happens

Trigger: validateInputFilePatternSupported / validateOutputFilePrefixSupported on a gs:// path whose bucket does not exist, is not readable by the caller, is misspelled, or the GCS call fails transiently.

Common situations: Typo in bucket or file path; input file deleted before launch; service account lacking storage.objects.get/list; running on Dataflow with a bucket in another project; requester-pays bucket without a user project.

Understand the failure class

Background: "File not found" and ENOENT errors: why libraries can't find a file that should exist — this error's family across 50 libraries.

Related errors

• Error constructing default value for gcpTempLocation: tempLo
• The key '%s' in GCS custom audit entries exceeds the %d-char
• The value '%s' in GCS custom audit entries exceeds the %d-ch
• The maximum allowed number of GCS custom audit entries (incl
• Unable create default bucket.

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/f055c83c44512037. Report an issue: GitHub.