apache/beam · error · TypeError

The "use_native_datetime" parameter cannot be True for EXPOR

Error message

The "use_native_datetime" parameter cannot be True for EXPORT. Please set the "use_native_datetime" parameter to False *OR* set the "method" parameter to ReadFromBigQuery.Method.DIRECT_READ.

What it means

ReadFromBigQuery's EXPORT method writes query/table results to GCS as Avro/JSON files, and the Avro path cannot preserve the native DATETIME logical type. The library raises TypeError at construction time when use_native_datetime=True is combined with method=EXPORT, telling you to either disable native datetime or switch to DIRECT_READ.

Source

Thrown at sdks/python/apache_beam/io/gcp/bigquery.py:3062

      method=None,
      use_native_datetime=False,
      output_type=None,
      timeout=None,
      *args,
      query_output_schema=None,
      **kwargs):
    self.method = method or ReadFromBigQuery.Method.EXPORT
    self.use_native_datetime = use_native_datetime
    self.output_type = output_type
    self.query_output_schema = query_output_schema
    self._args = args
    self._kwargs = kwargs
    if timeout is not None:
      self._kwargs['timeout'] = timeout

    if self.method == ReadFromBigQuery.Method.EXPORT \
        and self.use_native_datetime is True:
      raise TypeError(
          'The "use_native_datetime" parameter cannot be True for EXPORT.'
          ' Please set the "use_native_datetime" parameter to False *OR*'
          ' set the "method" parameter to ReadFromBigQuery.Method.DIRECT_READ.')

    if gcs_location and self.method == ReadFromBigQuery.Method.EXPORT:
      if not isinstance(gcs_location, (str, ValueProvider)):
        raise TypeError(
            '%s: gcs_location must be of type string'
            ' or ValueProvider; got %r instead' %
            (self.__class__.__name__, type(gcs_location)))
      if isinstance(gcs_location, str):
        gcs_location = StaticValueProvider(str, gcs_location)

    if self.output_type == 'BEAM_ROW' and self._kwargs.get('query',
                                                           None) is not None:
      if self.query_output_schema is None:
        raise ValueError(
            "Both a query and an output type of 'BEAM_ROW' were specified "

View on GitHub (pinned to 12126d8942)

Solutions

Set use_native_datetime=False in ReadFromBigQuery
Change method=ReadFromBigQuery.Method.DIRECT_READ (requires the BigQuery Storage Read API)
If DATETIME fidelity is not needed, drop the use_native_datetime parameter entirely so it defaults appropriately

Example fix

// before
ReadFromBigQuery(table='proj:ds.tbl', method=ReadFromBigQuery.Method.EXPORT, use_native_datetime=True)
// after
ReadFromBigQuery(table='proj:ds.tbl', method=ReadFromBigQuery.Method.DIRECT_READ, use_native_datetime=True)

Defensive patterns

Strategy: validation

Validate before calling

if method == ReadFromBigQuery.Method.EXPORT and use_native_datetime:
    raise ValueError('use_native_datetime requires DIRECT_READ')

Prevention

Only set use_native_datetime when using DIRECT_READ
Encapsulate ReadFromBigQuery construction in a helper that enforces valid option combos
Review Beam release notes when enabling new type options

When it happens

Trigger: Calling ReadFromBigQuery(..., method=ReadFromBigQuery.Method.EXPORT, use_native_datetime=True) in apache_beam/io/gcp/bigquery.py:3062; also leaving use_native_datetime=True (explicitly set) while choosing EXPORT to avoid streaming reads.

Common situations: Users migrating to Beam's newer native datetime support keep their existing EXPORT-based pipeline; copy-pasting a DIRECT_READ example but keeping EXPORT; explicitly enabling use_native_datetime for timezone-aware parsing without realizing it only works with the storage API read path.

Understand the failure class

Background: Conflicting config options: "cannot be used together" — configuration validation errors across open-source libraries — this error's family across 162 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/6243cf7550c25ec1. Report an issue: GitHub.