apache/beam · error · IllegalArgumentException

Expected a valid 'gs://' path but was given

Error message

Expected a valid 'gs://' path but was given '%s'

What it means

GcsPathValidator.getGcsPath converts a user-supplied path string to a GcsPath via GcsPath.fromUri. If the string is not a valid gs:// URI, fromUri throws IllegalArgumentException, which is rethrown with 'Expected a valid 'gs://' path but was given '%s''. It guards all GCS input/output validation entry points.

Solutions

1. Prefix the path with gs:// (full form gs://bucket/path)
2. Check the option/value being passed for accidental local-path defaults
3. Use GcsPath.fromUri in your own code to fail fast with the raw URI problem

Example fix

// before
String output = "/home/user/output/prefix";
// after
String output = "gs://my-bucket/output/prefix";

boolean isGcsPath(String s) {
  try { GcsPath.fromUri(s); return true; } catch (IllegalArgumentException e) { return false; }
}
// call before validate: if (!isGcsPath(output)) output = "gs://" + output;

boolean isGcsUri(String s) { return s != null && s.startsWith("gs://") && GcsPath.fromUri(s).getBucket() != null; }

try { GcsPath.fromUri(path); } catch (IllegalArgumentException e) { throw new IllegalArgumentException("Not a gs:// path: " + path, e); }

When it happens

Trigger: Passing a local filesystem path (e.g. /tmp/out or file:///tmp), a bare bucket name, or a malformed URI (gs:/bucket, gs://) to validateInputFilePatternSupported or validateOutputFilePrefixSupported.

Common situations: Config defaulting to local paths when running on Dataflow; missing 'gs://' scheme after string concatenation; Windows-style paths pasted into options.

Understand the failure class

Background: "Invalid URL" errors: why new URL(), URI.parse, and reqwest::Url reject your string — missing scheme, whitespace, and bad path format — this error's family across 39 libraries.

Related errors

• Could not find file
• Error constructing default value for gcpTempLocation…
• Output path does not exist or is not writeable
• stagingBucketName must end with '/'
• The key ' ' in GCS custom audit entries exceeds the…

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/b8ca296c37fd4f71. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/extensions/google-cloud-platform-core/src/main/java/org/apache/beam/sdk/extensions/gcp/storage/GcsPathValidator.java:93

        !gcsPath.getObject().contains("//"),
        "Dataflow Service does not allow objects with consecutive slashes");
    return gcsPath.toResourceName();
  }

  private void verifyPathIsAccessible(String path, String errorMessage) {
    GcsPath gcsPath = getGcsPath(path);
    try {
      gcpOptions.getGcsUtil().verifyBucketAccessible(gcsPath);
    } catch (IOException e) {
      throw new RuntimeException(String.format(errorMessage, path), e);
    }
  }

  private GcsPath getGcsPath(String path) {
    try {
      return GcsPath.fromUri(path);
    } catch (IllegalArgumentException e) {
      throw new IllegalArgumentException(
          String.format("Expected a valid 'gs://' path but was given '%s'", path), e);
    }
  }
}

View on GitHub (pinned to 12126d8942)