apache/beam · error · ValueError

Spanner required read operation, sql or table with columns.

Error message

Spanner required read operation, sql or table with columns.

What it means

ReadFromSpanner requires at least one source of read configuration: a ReadOperation (via read_operations), or a query (sql), or a table with columns. When none is supplied (and the transform is at the pipeline root), the library raises this ValueError because it has nothing to read.

Solutions

1. Pass a query: ReadFromSpanner(..., sql='SELECT * FROM table').
2. Or pass table and columns: ReadFromSpanner(..., table='mytable', columns=['col1','col2']).
3. Or pass read_operations=[ReadOperation(...)] when applying at pipeline root.
4. If reading per-element operations, pipe a PCollection of ReadOperation objects into SpannerRead instead of ReadFromSpanner.

Example fix

# before
p | ReadFromSpanner(project='p', instance='i', database='d')
# after
p | ReadFromSpanner(project='p', instance='i', database='d', sql='SELECT * FROM users')

def validate_read_config(sql=None, table=None, columns=None, read_operations=None):
    if not (sql or (table and columns) or read_operations):
        raise ValueError('ReadFromSpanner needs sql, or table+columns, or read_operations')
    return True

try:
    result = p | ReadFromSpanner(project=pr, instance=ins, database=db, sql=sql)
except ValueError as e:
    if 'required read operation' in str(e):
        logging.error('Missing read config for Spanner: %s', e)
    raise

When it happens

Trigger: Calling ReadFromSpanner(project=..., instance=..., database=...) with no sql, no table/columns, and no read_operations; or passing read_operations=None when not piping a PCollection of ReadOperation objects.

Common situations: Copy-pasting a ReadFromSpanner template and leaving the sql/table argument empty; building read config conditionally so all arguments end up None; mixing up SpannerRead (which takes PCollection of ReadOperation) with ReadFromSpanner (which needs explicit sql/table).

Understand the failure class

Background: "missing required argument" and "the following required arguments were not provided": what required-argument errors mean and how to fix them — this error's family across 20 libraries.

Related errors

• Columns are required with the table name.
• Invalid object type: . Object must be an instance of…
• No or more than one write mutation operation provided: <
• Read operation in the constructor only works with the root…
• Unknown operation action

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/6054005bc0cd7bcd. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/io/gcp/experimental/spannerio.py:776

                table=table, columns=columns, index=index, keyset=keyset)
        ]
      elif sql is not None:
        self._read_operations = [
            ReadOperation.query(
                sql=sql, params=params, param_types=param_types)
        ]

  def expand(self, pbegin):
    if self._read_operations is not None and isinstance(pbegin, PBegin):
      pcoll = pbegin.pipeline | Create(self._read_operations)
    elif not isinstance(pbegin, PBegin):
      if self._read_operations is not None:
        raise ValueError(
            "Read operation in the constructor only works with "
            "the root of the pipeline.")
      pcoll = pbegin
    else:
      raise ValueError(
          "Spanner required read operation, sql or table "
          "with columns.")

    if self._transaction is None:
      # reading as batch read using the spanner partitioning query to create
      # batches.
      p = (
          pcoll
          | 'Generate Partitions' >> ParDo(
              _CreateReadPartitions(spanner_configuration=self._configuration))
          | 'Reshuffle' >> Reshuffle()
          | 'Read From Partitions' >> ParDo(
              _ReadFromPartitionFn(spanner_configuration=self._configuration)))
    else:
      # reading as naive read, in which we don't make batches and execute the
      # queries as a single read.
      p = (
          pcoll

View on GitHub (pinned to 12126d8942)