apache/beam · error · SplitNotPossibleError

Query cannot have any inequality filters.

Error message

Query cannot have any inequality filters.

What it means

Apache Beam's Cloud Datastore query splitter refuses to compute split points for a query containing inequality filters ('<', '<=', '>', '>='). Inequality filters force results into a contiguous index range, so the scatter-query technique used to divide the query cannot produce balanced splits. The check runs in validate_split, invoked by get_splits before any splitting is attempted.

Solutions

  1. Remove inequality filters from the query before passing it to get_splits, or fetch the filtered subset by equality/keys and split that.
  2. Add an equality filter on the same property (Datastore requires one, e.g. status = 'active'), which keeps the query splittable.
  3. Pre-compute the boundary keys yourself and issue multiple equality/key-range queries manually instead of relying on get_splits.
  4. If the operator comes from a ValueProvider, validate the resolved operator early in the pipeline and fail with a clear user-facing message.

Example fix

// before
query = Query(kind='Person')
query.filters.append(('age', '>', 30))
splits = query_splitter.get_splits(client, query, num_splits=20)
// after
query = Query(kind='Person')
query.filters.append(('status', '=', 'active'))  # equality only
splits = query_splitter.get_splits(client, query, num_splits=20)
Defensive patterns

Strategy: validation

Validate before calling

def ensure_splittable(query):
    for prop, op in [(f[0], f[1].get() if hasattr(f[1], 'get') else f[1]) for f in query.filters]:
        if op in ('<', '<=', '>', '>='):
            raise ValueError(f'Inequality filter on {prop} prevents query splitting')

Try / catch

try:
    splits = query_splitter.get_splits(client, query, num_splits)
except SplitNotPossibleError as e:
    logging.warning('Query not splittable, running unsplit: %s', e)
    splits = [query]

Prevention

When it happens

Trigger: Calling query_splitter.get_splits(client, query, num_splits) where query.filters contains any filter whose operator is '<', '<=', '>', or '>=' (operators may come from a ValueProvider at runtime).

Common situations: Developers building Datastore export/backup pipelines pass user-built queries with range filters (e.g. timestamp > X) into Beam's split-for-parallelism helper; with template parameters the operator is only resolved from the ValueProvider at runtime, so the failure appears at pipeline runtime rather than construction time.

Understand the failure class

Background: UnsupportedOperationException and "is not supported" errors: when a library deliberately refuses a call — this error's family across 30 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/2268e2aea43677c7. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/io/gcp/datastore/v1new/query_splitter.py:109

  in inefficient sharding.

  Raises:
    QuerySplitterError if split could not be performed owing to query
      parameters.
  """
  if query.order:
    raise SplitNotPossibleError('Query cannot have any sort orders.')

  if query.limit is not None:
    raise SplitNotPossibleError('Query cannot have a limit set.')

  for filter in query.filters:
    if isinstance(filter[1], ValueProvider):
      filter_operator = filter[1].get()
    else:
      filter_operator = filter[1]
    if filter_operator in ['<', '<=', '>', '>=']:
      raise SplitNotPossibleError('Query cannot have any inequality filters.')


def _create_scatter_query(query, num_splits):
  """Creates a scatter query from the given user query."""
  # There is a split containing entities before and after each scatter entity:
  # ||---*------*------*------*------*------*------*---||  * = scatter entity
  # If we represent each split as a region before a scatter entity, there is an
  # extra region following the last scatter point. Thus, we do not need the
  # scatter entity for the last region.
  limit = (num_splits - 1) * KEYS_PER_SPLIT
  scatter_query = types.Query(
      kind=query.kind,
      project=query.project,
      namespace=query.namespace,
      order=[SCATTER_PROPERTY_NAME],
      projection=[KEY_PROPERTY_NAME],
      limit=limit)
  return scatter_query

View on GitHub (pinned to 12126d8942)