apache/beam · warning

Failed to translate Gql query

Error message

Failed to translate Gql query '{}'

What it means

DatastoreV1.translateGqlQueryWithLimitCheck logs 'Failed to translate Gql query' when translating a GQL query with a zero limit fails with DatastoreException INVALID_ARGUMENT. Because Datastore provides no way to detect whether a query already has a LIMIT clause, the connector assumes the user's query already contains a limit and retries translation without the zero limit.

Solutions

1. No action required — the connector automatically retries translating the original query without the zero limit.
2. Remove the explicit LIMIT from your GQL if you want the connector's limit/estimate logic to work as intended.
3. If translation still fails, fix the GQL syntax itself; the fallback re-raises non-INVALID_ARGUMENT errors unchanged.
4. Use withQuery (structured Query build) instead of GQL to avoid GQL translation entirely.

Example fix

// before
query = "SELECT * FROM MyKind LIMIT 100";
// after (recommended)
query = "SELECT * FROM MyKind"; // let the connector control limits
// or use a structured query:
Read read = DatastoreIO.v1().read().withQuery(com.google.datastore.v1.Query.newBuilder().addKindBuilder().setName("MyKind").build());

boolean hasLimitClause(String gql) { return java.util.regex.Pattern.compile("(?i)\\bLIMIT\\s+\\d+").matcher(gql).find(); }

try { q = translateGqlQueryWithLimitCheck(gql, datastore, projectId, db, ns, readTime); } catch (DatastoreException e) { if (e.getCode() != Code.INVALID_ARGUMENT) throw e; /* connector already retried without zero limit */ }

When it happens

Trigger: Calling DatastoreIO.v1().read().withGqlQuery(...) (via translateGqlQueryWithLimitCheck) on a GQL string that already includes a LIMIT clause; adding the sentinel 'LIMIT 0' makes the query invalid, so Datastore returns INVALID_ARGUMENT.

Common situations: Users providing GQL with their own LIMIT while the connector probes the result size with LIMIT 0; queries with syntax Datastore rejects for any INVALID_ARGUMENT reason (also triggers the same fallback).

Understand the failure class

Background: "Must be a positive integer", "Invalid value", "Unsupported": the invalid-argument-value error family, when a library rejects the value you pass — this error's family across 35 libraries.

Related errors

• A function must be provided to convert the input type into…
• A PValue contained in
• A schema was provided without a data format (or viceversa)…
• All inherited interfaces of
• An unsupported type of cache was passed in. Received

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/9b657cd9be3d00f9. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/datastore/DatastoreV1.java:619

        Datastore datastore,
        String projectId,
        String databaseId,
        String namespace,
        @Nullable Instant readTime)
        throws DatastoreException {
      String gqlQueryWithZeroLimit = gql + " LIMIT 0";
      try {
        Query translatedQuery =
            translateGqlQuery(
                gqlQueryWithZeroLimit, datastore, projectId, databaseId, namespace, readTime);
        // Clear the limit that we set.
        return translatedQuery.toBuilder().clearLimit().build();
      } catch (DatastoreException e) {
        // Note: There is no specific error code or message to detect if the query already has a
        // limit, so we just check for INVALID_ARGUMENT and assume that that the query might have
        // a limit already set.
        if (e.getCode() == Code.INVALID_ARGUMENT) {
          LOG.warn("Failed to translate Gql query '{}'", gqlQueryWithZeroLimit, e);
          LOG.warn("User query might have a limit already set, so trying without zero limit");
          // Retry without the zero limit.
          return translateGqlQuery(gql, datastore, projectId, databaseId, namespace, readTime);
        } else {
          throw e;
        }
      }
    }

    /** Translates a gql query string to {@link Query}. */
    private static Query translateGqlQuery(
        String gql,
        Datastore datastore,
        String projectId,
        String databaseId,
        String namespace,
        @Nullable Instant readTime)
        throws DatastoreException {

View on GitHub (pinned to 12126d8942)