apache/beam · error · ValueError

query.project cannot be empty

Error message

query.project cannot be empty

What it means

ReadFromDatastore's __init__ validates the query and raises ValueError when query.project is empty/None. The project id is required to talk to Cloud Datastore, so a query without it cannot be executed.

Source

Thrown at sdks/python/apache_beam/io/gcp/datastore/v1new/datastoreio.py:126

  _NUM_QUERY_SPLITS_MIN = 12
  # Default bundle size of 64MB.
  _DEFAULT_BUNDLE_SIZE_BYTES = 64 * 1024 * 1024

  def __init__(self, query, num_splits=0):
    """Initialize the `ReadFromDatastore` transform.

    This transform outputs elements of type
    :class:`~apache_beam.io.gcp.datastore.v1new.types.Entity`.

    Args:
      query: (:class:`~apache_beam.io.gcp.datastore.v1new.types.Query`) query
        used to fetch entities.
      num_splits: (:class:`int`) (optional) Number of splits for the query.
    """
    super().__init__()

    if not query.project:
      raise ValueError("query.project cannot be empty")
    if not query:
      raise ValueError("query cannot be empty")
    if num_splits < 0:
      raise ValueError("num_splits must be greater than or equal 0")

    self._project = query.project
    # using _namespace conflicts with DisplayData._namespace
    self._datastore_namespace = query.namespace
    self._query = query
    self._num_splits = num_splits

  def expand(self, pcoll):
    # This is a composite transform involves the following:
    #   1. Create a singleton of the user provided `query` and apply a ``ParDo``
    #   that splits the query into `num_splits` queries if possible.
    #
    #   If the value of `num_splits` is 0, the number of splits will be
    #   computed dynamically based on the size of the data for the `query`.

View on GitHub (pinned to 12126d8942)

Solutions

1. Set project when constructing the query: Query(project='my-project', kind='MyKind').
2. Pass project explicitly to ReadFromDatastore/Query from a validated config value.
3. Check that the environment variable or config supplying the project id is populated before building the query.

Example fix

// before
query = Query(kind='Person')  # project missing
ReadFromDatastore(query=query)

// after
query = Query(project='my-gcp-project', kind='Person')
ReadFromDatastore(query=query)

if not query.project:
    raise ValueError('set query.project before ReadFromDatastore')

def has_project(query):
    return bool(getattr(query, 'project', None))

try:
    read = ReadFromDatastore(query=query, num_splits=n)
except ValueError as e:
    if 'query.project' in str(e):
        query = Query(project=get_default_project(), kind=query.kind)
        read = ReadFromDatastore(query=query, num_splits=n)
    else:
        raise

When it happens

Trigger: Constructing a datastore.Query without setting project; passing a Query built where project was left as None by default; programmatically building queries where the project variable was an empty string.

Common situations: Config-driven pipelines where the GCP project env var is unset; reusing a Query object across projects and clearing fields; copy-pasted code omitting the project argument.

Understand the failure class

Background: "missing required argument" and "the following required arguments were not provided": what required-argument errors mean and how to fix them — this error's family across 20 libraries.

Related errors

• query cannot be empty
• num_splits must be greater than or equal 0
• Datastore total statistics unavailable.
• MatchContinuously interval must be positive.
• Invalid create disposition %s. Expecting %s

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/523baf46ffa99662. Report an issue: GitHub.