apache/beam · warning

Unable to parallelize the given query: {}

Error message

Unable to parallelize the given query: {}

What it means

During Read expansion, Beam tries to use the Datastore query splitter to divide the query into multiple sub-queries for parallel execution. If splitting fails for any reason (query not splittable, splitter unavailable, split limit too low, or any exception), the connector logs this warning and falls back to a single unsplit query, meaning the read will run with reduced (single-reader) parallelism.

Source

Thrown at sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/datastore/DatastoreV1.java:1064

        } else {
          estimatedNumSplits = numSplits;
        }

        LOG.info("Splitting the query into {} splits", estimatedNumSplits);
        List<Query> querySplits;
        try {
          querySplits =
              splitQuery(
                  options.getProjectId(),
                  options.getDatabaseId(),
                  query,
                  options.getNamespace(),
                  datastore,
                  querySplitter,
                  estimatedNumSplits,
                  readTime);
        } catch (Exception e) {
          LOG.warn("Unable to parallelize the given query: {}", query, e);
          querySplits = ImmutableList.of(query);
        }

        // assign unique keys to query splits.
        for (Query subquery : querySplits) {
          c.output(subquery);
        }
      }

      @Override
      public void populateDisplayData(DisplayData.Builder builder) {
        super.populateDisplayData(builder);
        builder.include("options", options);
        if (numSplits > 0) {
          builder.add(
              DisplayData.item("numQuerySplits", numSplits)
                  .withLabel("Requested number of Query splits"));
        }

View on GitHub (pinned to 12126d8942)

Solutions

1. Restructure the query so it is splittable: query a top-level Kind without restrictive inequality filters, or add an ancestor if appropriate.
2. Lower withNumQuerySplits() to a value the Cloud Datastore splitter supports (splitting may fail if requested splits are too granular for the data size).
3. Accept the warning if data volume is small; the read still completes with one reader.
4. Check the attached exception to see if the underlying split request failed due to permissions/quota and fix that instead.

Example fix

// before
datastoreIO.read().withQuery(query).withNumQuerySplits(10000)
// after
datastoreIO.read().withQuery(query).withNumQuerySplits(20)

// Java: prefer queries the splitter supports
if (query.hasFilter() && !isEqualityOnlyFilter(query.getFilter())) {
  LOG.info("Query may not be splittable; expect single-reader fallback");
}

When it happens

Trigger: DatastoreIO.read() on a query the QuerySplitter cannot split (e.g. query without an ancestor or with an inequality filter on a non-id field, or GQL-translated queries), estimatedNumSplits below minimum supported, or split requests throwing exceptions (quota/API errors).

Common situations: Users reading a single Kind with filters that prevent splitting; running with numQuerySplits set higher than the API allows; reads from namespaces or with readTime where splitter support is limited.

Understand the failure class

Background: UnsupportedOperationException and "is not supported" errors: when a library deliberately refuses a call — this error's family across 30 libraries.

Related errors

• User query might have a limit already set, so trying without
• Mutation {} does not have an operation type set.
• overloadRatio must be greater than 1.0
• throttleDelaySecs must be greater than 0
• requires an input Schema. Note that only Row or user classe

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/032471e21ac68c3b. Report an issue: GitHub.