{"record":{"id":"5dee4ed3be5ff1c1","repo":"apache/beam","slug":"bigquery-temp-location-expected-a-valid-gs-path-but-was","errorCode":null,"errorMessage":"BigQuery temp location expected a valid 'gs://' path, but was given '%s'","messagePattern":"BigQuery temp location expected a valid 'gs://' path, but was given '(.+?)'","errorType":"validation","errorClass":"IllegalArgumentException","httpStatus":null,"severity":"error","filePath":"sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BatchLoads.java","lineNumber":310,"sourceCode":"      if (!customGcsTempLocation.isAccessible()) {\n        // Can't perform verification in this case.\n        return;\n      } else if (Strings.isNullOrEmpty(customGcsTempLocation.get())) {\n        tempLocation = options.getTempLocation();\n      } else {\n        tempLocation = customGcsTempLocation.get();\n      }\n    }\n    checkArgument(\n        !Strings.isNullOrEmpty(tempLocation),\n        \"BigQueryIO.Write needs a GCS temp location to store temp files. \"\n            + \"This can be set by withCustomGcsTempLocation() in the Builder \"\n            + \"or through the fallback pipeline option --tempLocation.\");\n    if (bigQueryServices == null) {\n      try {\n        GcsPath.fromUri(tempLocation);\n      } catch (IllegalArgumentException e) {\n        throw new IllegalArgumentException(\n            String.format(\n                \"BigQuery temp location expected a valid 'gs://' path, but was given '%s'\",\n                tempLocation),\n            e);\n      }\n    }\n  }\n\n  // Expand the pipeline when the user has requested periodically-triggered file writes.\n  private WriteResult expandTriggered(PCollection<KV<DestinationT, ElementT>> input) {\n    Duration triggeringFrequency = Preconditions.checkStateNotNull(this.triggeringFrequency);\n    Pipeline p = input.getPipeline();\n    final PCollectionView<String> loadJobIdPrefixView = createJobIdPrefixView(p, JobType.LOAD);\n    final PCollectionView<String> tempLoadJobIdPrefixView =\n        createJobIdPrefixView(p, JobType.TEMP_TABLE_LOAD);\n    final PCollectionView<String> zeroLoadJobIdPrefixView =\n        createJobIdPrefixView(p, JobType.SCHEMA_UPDATE);\n    final PCollectionView<String> copyJobIdPrefixView = createJobIdPrefixView(p, JobType.COPY);","sourceCodeStart":292,"sourceCodeEnd":328,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BatchLoads.java#L292-L328","documentation":"BatchLoads.validate() requires the BigQuery sink's temp location to be a valid 'gs://' GCS URI whenever custom (non-builtin) BigQuery services are used. It parses tempLocation with GcsPath.fromUri and rethrows any IllegalArgumentException with a message telling the user to set it via withCustomGcsTempLocation() or the --tempLocation pipeline option.","triggerScenarios":"Calling BigQueryIO.write() with withCustomGcsTempLocation(\"/local/path\") or a non-GCS URI (http://, s3://, plain path), especially with custom bigQueryServices","commonSituations":"Setting --tempLocation to a non-GCS value (local dir or another cloud's URI) while running with BigQuery Storage API or custom services; forgetting to set tempLocation at all in test harnesses using testBigQueryServices.","solutions":["Set --tempLocation=gs://<bucket>/<path> in your pipeline options","Or call BigQueryIO.write().withCustomGcsTempLocation(new Validation.GCSPath(\"gs://bucket/tmp\"))","Remove the invalid local/file URI and use a real GCS bucket that the pipeline's credentials can write to","If running on Dataflow, ensure the service account has storage.objects.create on the temp bucket"],"exampleFix":"// before\n.apply(\"WriteBigQuery\", BigQueryIO.writeTableRows().to(table)\n    .withCustomGcsTempLocation(new Validation.GCSPath(\"/tmp/beam\")));\n// after\n.apply(\"WriteBigQuery\", BigQueryIO.writeTableRows().to(table)\n    .withCustomGcsTempLocation(new Validation.GCSPath(\"gs://my-bucket/tmp\")));","handlingStrategy":"validation","validationCode":"// Java\nString tempLocation = options.getTempLocation();\nif (tempLocation == null || !tempLocation.startsWith(\"gs://\")) {\n  throw new IllegalArgumentException(\"--tempLocation must be a gs:// path, got: \" + tempLocation);\n}","typeGuard":"// Java\nboolean isGcs = tempLocation != null && tempLocation.startsWith(\"gs://\");","tryCatchPattern":"// Java\ntry {\n  GcsPath.fromUri(tempLocation);\n} catch (IllegalArgumentException e) {\n  throw new IllegalArgumentException(\"Set --tempLocation to a gs:// URI, got: \" + tempLocation, e);\n}","preventionTips":["Always pass --tempLocation=gs://<bucket>/<path> when running pipelines with BigQuery sinks","Prefer withCustomGcsTempLocation() with an explicit GCS path in code","Ensure the runner's default temp location isn't a local directory","Grant the pipeline service account write access to the temp bucket"],"tags":["java","bigquery","gcs","configuration"],"backgroundTag":"invalid-url-format","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}