apache/beam · error · TypeError

%s: gcs_location must be of type string or ValueProvider; go

Error message

%s: gcs_location must be of type string or ValueProvider; got %r instead

What it means

When using ReadFromBigQuery with the EXPORT method, gcs_location identifies the GCS path for export files and must be a plain string or a Beam ValueProvider. The library raises TypeError in __init__ (bigquery.py:3069) for any other type so misconfiguration fails fast before pipeline submission.

Source

Thrown at sdks/python/apache_beam/io/gcp/bigquery.py:3069

    self.method = method or ReadFromBigQuery.Method.EXPORT
    self.use_native_datetime = use_native_datetime
    self.output_type = output_type
    self.query_output_schema = query_output_schema
    self._args = args
    self._kwargs = kwargs
    if timeout is not None:
      self._kwargs['timeout'] = timeout

    if self.method == ReadFromBigQuery.Method.EXPORT \
        and self.use_native_datetime is True:
      raise TypeError(
          'The "use_native_datetime" parameter cannot be True for EXPORT.'
          ' Please set the "use_native_datetime" parameter to False *OR*'
          ' set the "method" parameter to ReadFromBigQuery.Method.DIRECT_READ.')

    if gcs_location and self.method == ReadFromBigQuery.Method.EXPORT:
      if not isinstance(gcs_location, (str, ValueProvider)):
        raise TypeError(
            '%s: gcs_location must be of type string'
            ' or ValueProvider; got %r instead' %
            (self.__class__.__name__, type(gcs_location)))
      if isinstance(gcs_location, str):
        gcs_location = StaticValueProvider(str, gcs_location)

    if self.output_type == 'BEAM_ROW' and self._kwargs.get('query',
                                                           None) is not None:
      if self.query_output_schema is None:
        raise ValueError(
            "Both a query and an output type of 'BEAM_ROW' were specified "
            "without a query_output_schema. When using a query, you must "
            "provide query_output_schema so the output schema can be "
            "determined without reading an existing table. The schema should "
            "be a BigQuery schema dict, e.g. "
            "{'fields': [{'name': 'col', 'type': 'STRING', 'mode': 'NULLABLE'}"
            ", ...]}, or a TableSchema object.")

View on GitHub (pinned to 12126d8942)

Solutions

Convert the location to a string: gcs_location='gs://my-bucket/tmp'
Wrap runtime values in StaticValueProvider(str, gcs_location) or RuntimeValueProvider
If using pathlib.Path, call str(path) before passing

Example fix

// before
from pathlib import Path
ReadFromBigQuery(table=t, gcs_location=Path('gs://bucket/tmp'))
// after
ReadFromBigQuery(table=t, gcs_location='gs://bucket/tmp')

Defensive patterns

Strategy: type-guard

Validate before calling

assert isinstance(gcs_location, (str, ValueProvider)), type(gcs_location)

Type guard

def is_valid_gcs_location(x):
    return isinstance(x, (str, ValueProvider))

Try / catch

try:
    transform = ReadFromBigQuery(..., gcs_location=gcs)
except TypeError as e:
    if 'gcs_location' in str(e):
        gcs = str(gcs)

Prevention

Always pass gs:// URIs as plain strings
Wrap runtime/template values in ValueProvider subclasses
Never pass Path or client-library objects where strings are documented

When it happens

Trigger: Passing gcs_location as a list, dict, pathlib.Path, gs:// URI object, None-wrapped custom type, or any non-str/non-ValueProvider while method=EXPORT.

Common situations: Using pathlib.Path('gs://bucket/tmp') instead of its str form; passing a google.cloud.storage Bucket/Blob object; wrapping the location in a container; dynamic templates need ValueProvider but users pass a runtime-parameterized custom callable.

Understand the failure class

Background: Type mismatch errors: IllegalArgumentException, TypeError and type guards across 150 open-source libraries — this error's family across 150 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/ddcc235055b25bea. Report an issue: GitHub.