apache/beam · error · ValueError

query cannot be empty

Error message

query cannot be empty

What it means

Raised by the argument-validation guard in ReadFromDatastore.__init__ (Datastore v1new source): after the helper checks that query.project is set, this generic empty-object guard rejects a falsy `query` argument — i.e. the caller passed None or an otherwise empty Query object instead of a fully-formed apache_beam.io.gcp.datastore.v1new.types.Query. The input at fault is the `query` parameter itself; construct a Query with a valid kind/project before passing it to the transform.

Source

Thrown at sdks/python/apache_beam/io/gcp/datastore/v1new/datastoreio.py:128

  _DEFAULT_BUNDLE_SIZE_BYTES = 64 * 1024 * 1024

  def __init__(self, query, num_splits=0):
    """Initialize the `ReadFromDatastore` transform.

    This transform outputs elements of type
    :class:`~apache_beam.io.gcp.datastore.v1new.types.Entity`.

    Args:
      query: (:class:`~apache_beam.io.gcp.datastore.v1new.types.Query`) query
        used to fetch entities.
      num_splits: (:class:`int`) (optional) Number of splits for the query.
    """
    super().__init__()

    if not query.project:
      raise ValueError("query.project cannot be empty")
    if not query:
      raise ValueError("query cannot be empty")
    if num_splits < 0:
      raise ValueError("num_splits must be greater than or equal 0")

    self._project = query.project
    # using _namespace conflicts with DisplayData._namespace
    self._datastore_namespace = query.namespace
    self._query = query
    self._num_splits = num_splits

  def expand(self, pcoll):
    # This is a composite transform involves the following:
    #   1. Create a singleton of the user provided `query` and apply a ``ParDo``
    #   that splits the query into `num_splits` queries if possible.
    #
    #   If the value of `num_splits` is 0, the number of splits will be
    #   computed dynamically based on the size of the data for the `query`.
    #
    #   2. The resulting ``PCollection`` is sharded across workers using a

View on GitHub (pinned to 12126d8942)

Solutions

  1. Ensure a fully populated Query (project and kind set) is passed to ReadFromDatastore.
  2. Guard the construction site: raise/validate that the query has project and kind before building the transform.
  3. Fix config loading so query fields are actually populated from the source configuration.

Example fix

// before
query = None
ReadFromDatastore(query=query)  # ValueError: query cannot be empty

// after
query = Query(project='my-gcp-project', kind='Person')
ReadFromDatastore(query=query)
Defensive patterns

Strategy: validation

Validate before calling

if not query:
    raise ValueError('query must be populated with project and kind')
if not query.kind:
    raise ValueError('query.kind must be set')

Type guard

def is_usable_query(query):
    return bool(query) and bool(getattr(query, 'kind', None))

Try / catch

try:
    read = ReadFromDatastore(query=query)
except ValueError as e:
    if 'query cannot be empty' in str(e):
        query = build_query_from_config(cfg)  # re-derive the query
        read = ReadFromDatastore(query=query)
    else:
        raise

Prevention

When it happens

Trigger: Passing None in place of a query; constructing datastore.Query() with neither project nor kind set so it evaluates falsy (e.g. no kind attribute set); passing an empty placeholder query from config parsing.

Common situations: Pipeline templates where query parameters come from a config file that failed to populate; dynamically-built queries where all optional fields were omitted; typo passing the wrong variable (e.g. query=None).

Understand the failure class

Background: "must not be empty", "cannot be empty" — required-field validation errors across open-source libraries — this error's family across 41 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/0e4afe95cdc68ac1. Report an issue: GitHub.