apache/beam · error · IllegalArgumentException

BigQuery temp location expected a valid 'gs://' path, but wa

Error message

BigQuery temp location expected a valid 'gs://' path, but was given '%s'

What it means

BigQueryIO.Read.validate checks pipeline options before running: for non-DIRECT_READ methods the job needs a GCS temp location (BigQueryOptions.getTempLocation()) to stage intermediate results. An empty, null, or non-'gs://' tempLocation fails this checkArgument — the pipeline cannot read from BigQuery without valid GCS temp storage.

Source

Thrown at sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryIO.java:1443

    @Override
    public void validate(@Nullable PipelineOptions maybeOptions) {
      PipelineOptions options = checkArgumentNotNull(maybeOptions);
      // Even if existence validation is disabled, we need to make sure that the BigQueryIO
      // read is properly specified.
      BigQueryOptions bqOptions = options.as(BigQueryOptions.class);

      if (getMethod() != TypedRead.Method.DIRECT_READ) {
        String tempLocation = bqOptions.getTempLocation();
        checkArgument(
            !Strings.isNullOrEmpty(tempLocation),
            "BigQueryIO.Read needs a GCS temp location to store temp files."
                + "This can be set with option --tempLocation.");
        if (getBigQueryServices() == null) {
          try {
            GcsPath.fromUri(tempLocation);
          } catch (IllegalArgumentException e) {
            throw new IllegalArgumentException(
                String.format(
                    "BigQuery temp location expected a valid 'gs://' path, but was given '%s'",
                    tempLocation),
                e);
          }
        }
        checkArgument(
            getBadRecordRouter().equals(BadRecordRouter.THROWING_ROUTER),
            "BigQueryIO Read with Error Handling is only available when DIRECT_READ is used");
      }

      ValueProvider<TableReference> table = getTableProvider();
      ValueProvider<String> query = getQuery();

      // Note that a table or query check can fail if the table or dataset are created by
      // earlier stages of the pipeline or if a query depends on earlier stages of a pipeline.
      // For these cases the withoutValidation method can be used to disable the check.
      if (getValidate()) {

View on GitHub (pinned to 12126d8942)

Solutions

Set --tempLocation to a valid GCS path, e.g. gs://my-bucket/temp
Verify the URI parses as a GcsPath (starts with gs:// and has a bucket)
If running on a runner that supplies tempLocation automatically, confirm it is a GCS path

Example fix

// before
--tempLocation=/tmp/beam-temp
// after
--tempLocation=gs://my-bucket/beam-temp

Defensive patterns

Strategy: validation

Validate before calling

// Before running
String tempLocation = options.getTempLocation();
if (tempLocation == null || !tempLocation.startsWith("gs://")) {
  throw new IllegalArgumentException("--tempLocation must be a gs:// path, got: " + tempLocation);
}

Prevention

Always pass --tempLocation=gs://bucket/path when using BigQueryIO
Validate the URI with GcsPath.fromUri() in a smoke test
Do not point tempLocation at local or non-GCS filesystems

When it happens

Trigger: Running BigQueryIO read/write with --tempLocation set to a non-GCS URI (e.g. local path, s3://, or malformed gs:// URI) while using the default BigQueryServices (not injected test stubs).

Common situations: Forgetting --tempLocation and picking up a wrong default; pointing tempLocation at a local directory; typos like 'gs:/bucket' or 'file:///tmp'.

Understand the failure class

Background: "Invalid URL" errors: why new URL(), URI.parse, and reqwest::Url reject your string — missing scheme, whitespace, and bad path format — this error's family across 39 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/47cf4fb43f71e04b. Report an issue: GitHub.