{"record":{"id":"2268e2aea43677c7","repo":"apache/beam","slug":"query-cannot-have-any-inequality-filters","errorCode":null,"errorMessage":"Query cannot have any inequality filters.","messagePattern":"Query cannot have any inequality filters\\.","errorType":"exception","errorClass":"SplitNotPossibleError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/io/gcp/datastore/v1new/query_splitter.py","lineNumber":109,"sourceCode":"  in inefficient sharding.\n\n  Raises:\n    QuerySplitterError if split could not be performed owing to query\n      parameters.\n  \"\"\"\n  if query.order:\n    raise SplitNotPossibleError('Query cannot have any sort orders.')\n\n  if query.limit is not None:\n    raise SplitNotPossibleError('Query cannot have a limit set.')\n\n  for filter in query.filters:\n    if isinstance(filter[1], ValueProvider):\n      filter_operator = filter[1].get()\n    else:\n      filter_operator = filter[1]\n    if filter_operator in ['<', '<=', '>', '>=']:\n      raise SplitNotPossibleError('Query cannot have any inequality filters.')\n\n\ndef _create_scatter_query(query, num_splits):\n  \"\"\"Creates a scatter query from the given user query.\"\"\"\n  # There is a split containing entities before and after each scatter entity:\n  # ||---*------*------*------*------*------*------*---||  * = scatter entity\n  # If we represent each split as a region before a scatter entity, there is an\n  # extra region following the last scatter point. Thus, we do not need the\n  # scatter entity for the last region.\n  limit = (num_splits - 1) * KEYS_PER_SPLIT\n  scatter_query = types.Query(\n      kind=query.kind,\n      project=query.project,\n      namespace=query.namespace,\n      order=[SCATTER_PROPERTY_NAME],\n      projection=[KEY_PROPERTY_NAME],\n      limit=limit)\n  return scatter_query","sourceCodeStart":91,"sourceCodeEnd":127,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/io/gcp/datastore/v1new/query_splitter.py#L91-L127","documentation":"Apache Beam's Cloud Datastore query splitter refuses to compute split points for a query containing inequality filters ('<', '<=', '>', '>='). Inequality filters force results into a contiguous index range, so the scatter-query technique used to divide the query cannot produce balanced splits. The check runs in validate_split, invoked by get_splits before any splitting is attempted.","triggerScenarios":"Calling query_splitter.get_splits(client, query, num_splits) where query.filters contains any filter whose operator is '<', '<=', '>', or '>=' (operators may come from a ValueProvider at runtime).","commonSituations":"Developers building Datastore export/backup pipelines pass user-built queries with range filters (e.g. timestamp > X) into Beam's split-for-parallelism helper; with template parameters the operator is only resolved from the ValueProvider at runtime, so the failure appears at pipeline runtime rather than construction time.","solutions":["Remove inequality filters from the query before passing it to get_splits, or fetch the filtered subset by equality/keys and split that.","Add an equality filter on the same property (Datastore requires one, e.g. status = 'active'), which keeps the query splittable.","Pre-compute the boundary keys yourself and issue multiple equality/key-range queries manually instead of relying on get_splits.","If the operator comes from a ValueProvider, validate the resolved operator early in the pipeline and fail with a clear user-facing message."],"exampleFix":"// before\nquery = Query(kind='Person')\nquery.filters.append(('age', '>', 30))\nsplits = query_splitter.get_splits(client, query, num_splits=20)\n// after\nquery = Query(kind='Person')\nquery.filters.append(('status', '=', 'active'))  # equality only\nsplits = query_splitter.get_splits(client, query, num_splits=20)","handlingStrategy":"validation","validationCode":"def ensure_splittable(query):\n    for prop, op in [(f[0], f[1].get() if hasattr(f[1], 'get') else f[1]) for f in query.filters]:\n        if op in ('<', '<=', '>', '>='):\n            raise ValueError(f'Inequality filter on {prop} prevents query splitting')","typeGuard":null,"tryCatchPattern":"try:\n    splits = query_splitter.get_splits(client, query, num_splits)\nexcept SplitNotPossibleError as e:\n    logging.warning('Query not splittable, running unsplit: %s', e)\n    splits = [query]","preventionTips":["Use equality filters in queries destined for get_splits","Resolve ValueProvider operators during template validation","Split data by key ranges yourself when range filters are unavoidable"],"tags":["google-cloud-datastore","query-validation","apache-beam"],"backgroundTag":"unsupported-operation","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}