{"record":{"id":"6cb6c4934a7acc22","repo":"apache/beam","slug":"invalid-gcs-location-r-writing-to-bigquery-with-file-loads-6cb6c4","errorCode":null,"errorMessage":"Invalid GCS location: %r.\nWriting to BigQuery with FILE_LOADS method requires a GCS location to be provided to write files to be loaded into BigQuery. Please provide a GCS bucket, or pass method=\"STREAMING_INSERTS\" to WriteToBigQuery.","messagePattern":"Invalid GCS location: %r\\.\nWriting to BigQuery with FILE_LOADS method requires a GCS location to be provided to write files to be loaded into BigQuery\\. Please provide a GCS bucket, or pass method=\"STREAMING_INSERTS\" to WriteToBigQuery\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/io/gcp/bigquery_file_loads.py","lineNumber":1022,"sourceCode":"    # thus we will need temporary tables for atomicity.\n    self.dynamic_destinations = bool(callable(destination))\n\n    self.additional_bq_parameters = additional_bq_parameters or {}\n    self.table_side_inputs = table_side_inputs or ()\n    self.schema_side_inputs = schema_side_inputs or ()\n\n    self.is_streaming_pipeline = is_streaming_pipeline\n    self.load_job_project_id = load_job_project_id\n    self._validate = validate\n    if self._validate:\n      self.verify()\n\n  def verify(self):\n    if (isinstance(self._custom_gcs_temp_location.get(), vp.StaticValueProvider)\n        and not self._custom_gcs_temp_location.get().startswith('gs://')):\n      # Only fail if the custom location is provided, and it is not a GCS\n      # location.\n      raise ValueError(\n          'Invalid GCS location: %r.\\n'\n          'Writing to BigQuery with FILE_LOADS method requires a '\n          'GCS location to be provided to write files to be '\n          'loaded into BigQuery. Please provide a GCS bucket, or '\n          'pass method=\"STREAMING_INSERTS\" to WriteToBigQuery.' %\n          self._custom_gcs_temp_location.get())\n    if self.is_streaming_pipeline and not self.triggering_frequency:\n      raise ValueError(\n          'triggering_frequency must be specified to use file'\n          'loads in streaming')\n    elif not self.is_streaming_pipeline and self.triggering_frequency:\n      raise ValueError(\n          'triggering_frequency can only be used with file'\n          'loads in streaming')\n    if not self.is_streaming_pipeline and self.with_auto_sharding:\n      return ValueError(\n          'with_auto_sharding can only be used with file loads in streaming.')\n","sourceCodeStart":1004,"sourceCodeEnd":1040,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py#L1004-L1040","documentation":"After resolving custom_gcs_temp_location, BigQueryBatchFileLoads verifies that a statically-provided location is a valid GCS URI ('gs://...'). If the value was provided but does not point at GCS, verify() raises this ValueError at graph-construction time.","triggerScenarios":"Passing custom_gcs_temp_location='/tmp/staging' or 's3://bucket' (not starting with gs://) as a StaticValueProvider to WriteToBigQuery with FILE_LOADS.","commonSituations":"Using a local temp directory in tests, copying an S3 path from another pipeline, or forgetting the 'gs://' scheme on a bucket name.","solutions":["Set custom_gcs_temp_location to a full 'gs://bucket/path' URI.","Use --temp_location gs://bucket/path instead and omit custom_gcs_temp_location.","Switch to method='STREAMING_INSERTS' if GCS staging is impossible."],"exampleFix":"// before\nbeam.io.WriteToBigQuery(table, custom_gcs_temp_location='/tmp/staging')\n// after\nbeam.io.WriteToBigQuery(table, custom_gcs_temp_location='gs://my-bucket/staging')","handlingStrategy":"validation","validationCode":"loc = custom_gcs_temp_location\nif loc is not None and not str(loc).startswith('gs://'):\n    raise ValueError(f'custom_gcs_temp_location must be a gs:// URI, got {loc!r}')","typeGuard":"def is_gcs_uri(v) -> bool:\n    return isinstance(v, str) and v.startswith('gs://')","tryCatchPattern":"try:\n    transform = beam.io.WriteToBigQuery(table, custom_gcs_temp_location=loc)\nexcept ValueError as e:\n    if 'Invalid GCS location' in str(e):\n        fix_temp_location()","preventionTips":["Validate bucket URIs against a gs:// prefix regex before constructing transforms","Avoid local filesystem paths for staging","Use pipeline-level --temp_location as a single source of truth"],"tags":["gcp","bigquery","gcs","validation"],"backgroundTag":"invalid-url-format","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T21:17:11.552Z"}