apache/beam · error · ValueError

A BigQuery table or a query must be specified

Error message

A BigQuery table or a query must be specified

What it means

Validation guard in BigQuerySource.__init__ (the legacy batch BigQuery source): the source must know where to read from, either a table reference or a SQL query, and neither was provided. It fires when callers construct the source directly (or via Read(BigQuerySource(...))) leaving both `table` and `query` as None, typically when options are built dynamically and the argument is dropped. Note the sibling check on the same lines rejects specifying both at once.

Source

Thrown at sdks/python/apache_beam/io/gcp/bigquery.py:725

      validate=False,
      pipeline_options=None,
      coder=None,
      use_standard_sql=False,
      flatten_results=True,
      kms_key=None,
      bigquery_job_labels=None,
      use_json_exports=False,
      job_name=None,
      step_name=None,
      unique_id=None,
      temp_dataset=None,
      query_priority=BigQueryQueryPriority.BATCH):
    if table is not None and query is not None:
      raise ValueError(
          'Both a BigQuery table and a query were specified.'
          ' Please specify only one of these.')
    elif table is None and query is None:
      raise ValueError('A BigQuery table or a query must be specified')
    elif table is not None:
      self.table_reference = bigquery_tools.parse_table_reference(
          table, dataset, project)
      self.query = None
      self.use_legacy_sql = True
    else:
      if isinstance(query, str):
        query = StaticValueProvider(str, query)
      self.query = query
      # TODO(BEAM-1082): Change the internal flag to be standard_sql
      self.use_legacy_sql = not use_standard_sql
      self.table_reference = None

    self.method = method
    self.gcs_location = gcs_location
    self.project = project
    self.validate = validate
    self.flatten_results = flatten_results

View on GitHub (pinned to 12126d8942)

Solutions

1. Provide a table: ReadFromBigQuery(table='project:dataset.table')
2. Or provide a query: ReadFromBigQuery(query='SELECT ...')
3. Check that config/env values feeding these arguments are actually set and non-empty

Example fix

// before
source = BigQuerySource(table=None, query=None)
// after
source = BigQuerySource(table='my-project:my_dataset.my_table')

if table is None and query is None:
    raise ValueError('A BigQuery table or a query must be specified')

def has_source(table, query):
    return table is not None or query is not None

try:
    source = BigQuerySource(table=table, query=query)
except ValueError as e:
    if 'must be specified' in str(e):
        table = os.environ['BIGQUERY_TABLE']
        source = BigQuerySource(table=table, query=query)
    else:
        raise

When it happens

Trigger: Constructing BigQuerySource() or ReadFromBigQuery() with neither table nor query (or both explicitly None).

Common situations: Config placeholders left empty (empty strings coerced to None), environment variables missing so table names resolve to None.

Understand the failure class

Background: "missing required argument" and "the following required arguments were not provided": what required-argument errors mean and how to fix them — this error's family across 20 libraries.

Related errors

• The TableRowJsonCoder requires a table schema for encoding o
• When using 'query' in ReadFromBigQuery YAML transform, 'sche
• A function must be provided to convert the input type into a
• You have to supply one of 'by' and 'level'
• Please specify a BigQuery table to read from.

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/c467b0db45144cf5. Report an issue: GitHub.