apache/beam · error · ValueError

Both a query and an output type of 'BEAM_ROW' were specified

Error message

Both a query and an output type of 'BEAM_ROW' were specified without a query_output_schema. When using a query, you must provide query_output_schema so the output schema can be determined without reading an existing table. The schema should be a BigQuery schema dict, e.g. {'fields': [{'name': 'col', 'type': 'STRING', 'mode': 'NULLABLE'}, ...]}, or a TableSchema object.

What it means

With output_type='BEAM_ROW', ReadFromBigQuery must produce a Beam schema for the output PCollection. When the input is a query, the schema cannot be inferred from an existing table, so query_output_schema is mandatory; ValueError is raised in __init__ (bigquery.py:3079) when it is missing.

Source

Thrown at sdks/python/apache_beam/io/gcp/bigquery.py:3079

        and self.use_native_datetime is True:
      raise TypeError(
          'The "use_native_datetime" parameter cannot be True for EXPORT.'
          ' Please set the "use_native_datetime" parameter to False *OR*'
          ' set the "method" parameter to ReadFromBigQuery.Method.DIRECT_READ.')

    if gcs_location and self.method == ReadFromBigQuery.Method.EXPORT:
      if not isinstance(gcs_location, (str, ValueProvider)):
        raise TypeError(
            '%s: gcs_location must be of type string'
            ' or ValueProvider; got %r instead' %
            (self.__class__.__name__, type(gcs_location)))
      if isinstance(gcs_location, str):
        gcs_location = StaticValueProvider(str, gcs_location)

    if self.output_type == 'BEAM_ROW' and self._kwargs.get('query',
                                                           None) is not None:
      if self.query_output_schema is None:
        raise ValueError(
            "Both a query and an output type of 'BEAM_ROW' were specified "
            "without a query_output_schema. When using a query, you must "
            "provide query_output_schema so the output schema can be "
            "determined without reading an existing table. The schema should "
            "be a BigQuery schema dict, e.g. "
            "{'fields': [{'name': 'col', 'type': 'STRING', 'mode': 'NULLABLE'}"
            ", ...]}, or a TableSchema object.")

    self.gcs_location = gcs_location
    self.bigquery_dataset_labels = {
        'type': 'bq_direct_read_' + str(uuid.uuid4())[0:10]
    }

  def expand(self, pcoll):
    if self.method == ReadFromBigQuery.Method.EXPORT:
      output_pcollection = self._expand_export(pcoll)
    elif self.method == ReadFromBigQuery.Method.DIRECT_READ:
      output_pcollection = self._expand_direct_read(pcoll)

View on GitHub (pinned to 12126d8942)

Solutions

1. Pass query_output_schema as a BigQuery schema dict, e.g. {'fields': [{'name': 'col', 'type': 'STRING', 'mode': 'NULLABLE'}]}
2. Or pass a TableSchema object matching the query's output columns
3. Or use output_type='PYTHON_DICT' if typed rows are not needed

Example fix

// before
ReadFromBigQuery(query='SELECT name FROM ds.tbl', output_type='BEAM_ROW')
// after
ReadFromBigQuery(query='SELECT name FROM ds.tbl', output_type='BEAM_ROW', query_output_schema={'fields': [{'name': 'name', 'type': 'STRING', 'mode': 'NULLABLE'}]})

if query and output_type == 'BEAM_ROW' and query_output_schema is None:
    raise ValueError('query_output_schema is required with query + BEAM_ROW')

try:
    beam.io.ReadFromBigQuery(query=q, output_type='BEAM_ROW', ...)
except ValueError as e:
    if 'query_output_schema' in str(e):
        # supply schema and retry
        ...

When it happens

Trigger: ReadFromBigQuery(query='SELECT ...', output_type='BEAM_ROW') without passing query_output_schema; using the default BEAM_ROW-ish output path with a query and only specifying selected_fields but no schema.

Common situations: Switching from a table input to a query while keeping output_type='BEAM_ROW'; building typed pipelines where users assume the schema is inferred from the SQL; copy-pasted examples that only set query.

Understand the failure class

Background: "missing required argument" and "the following required arguments were not provided": what required-argument errors mean and how to fix them — this error's family across 20 libraries.

Related errors

• Unexpected schema argument: %s.
• Unknown BigQuery field mode: {}
• Table schema must be of the type bigquery.TableSchema
• Unexpected schema argument: %s.
• Converting BigQuery type [{field_type}] to Python Beam type

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/384e0beeeb37af00. Report an issue: GitHub.