{"record":{"id":"9b657cd9be3d00f9","repo":"apache/beam","slug":"failed-to-translate-gql-query","errorCode":null,"errorMessage":"Failed to translate Gql query '{}'","messagePattern":"Failed to translate Gql query '(.+?)'","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/datastore/DatastoreV1.java","lineNumber":619,"sourceCode":"        Datastore datastore,\n        String projectId,\n        String databaseId,\n        String namespace,\n        @Nullable Instant readTime)\n        throws DatastoreException {\n      String gqlQueryWithZeroLimit = gql + \" LIMIT 0\";\n      try {\n        Query translatedQuery =\n            translateGqlQuery(\n                gqlQueryWithZeroLimit, datastore, projectId, databaseId, namespace, readTime);\n        // Clear the limit that we set.\n        return translatedQuery.toBuilder().clearLimit().build();\n      } catch (DatastoreException e) {\n        // Note: There is no specific error code or message to detect if the query already has a\n        // limit, so we just check for INVALID_ARGUMENT and assume that that the query might have\n        // a limit already set.\n        if (e.getCode() == Code.INVALID_ARGUMENT) {\n          LOG.warn(\"Failed to translate Gql query '{}'\", gqlQueryWithZeroLimit, e);\n          LOG.warn(\"User query might have a limit already set, so trying without zero limit\");\n          // Retry without the zero limit.\n          return translateGqlQuery(gql, datastore, projectId, databaseId, namespace, readTime);\n        } else {\n          throw e;\n        }\n      }\n    }\n\n    /** Translates a gql query string to {@link Query}. */\n    private static Query translateGqlQuery(\n        String gql,\n        Datastore datastore,\n        String projectId,\n        String databaseId,\n        String namespace,\n        @Nullable Instant readTime)\n        throws DatastoreException {","sourceCodeStart":601,"sourceCodeEnd":637,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/datastore/DatastoreV1.java#L601-L637","documentation":"DatastoreV1.translateGqlQueryWithLimitCheck logs 'Failed to translate Gql query' when translating a GQL query with a zero limit fails with DatastoreException INVALID_ARGUMENT. Because Datastore provides no way to detect whether a query already has a LIMIT clause, the connector assumes the user's query already contains a limit and retries translation without the zero limit.","triggerScenarios":"Calling DatastoreIO.v1().read().withGqlQuery(...) (via translateGqlQueryWithLimitCheck) on a GQL string that already includes a LIMIT clause; adding the sentinel 'LIMIT 0' makes the query invalid, so Datastore returns INVALID_ARGUMENT.","commonSituations":"Users providing GQL with their own LIMIT while the connector probes the result size with LIMIT 0; queries with syntax Datastore rejects for any INVALID_ARGUMENT reason (also triggers the same fallback).","solutions":["No action required — the connector automatically retries translating the original query without the zero limit.","Remove the explicit LIMIT from your GQL if you want the connector's limit/estimate logic to work as intended.","If translation still fails, fix the GQL syntax itself; the fallback re-raises non-INVALID_ARGUMENT errors unchanged.","Use withQuery (structured Query build) instead of GQL to avoid GQL translation entirely."],"exampleFix":"// before\nquery = \"SELECT * FROM MyKind LIMIT 100\";\n// after (recommended)\nquery = \"SELECT * FROM MyKind\"; // let the connector control limits\n// or use a structured query:\nRead read = DatastoreIO.v1().read().withQuery(com.google.datastore.v1.Query.newBuilder().addKindBuilder().setName(\"MyKind\").build());","handlingStrategy":"fallback","validationCode":"boolean hasLimitClause(String gql) { return java.util.regex.Pattern.compile(\"(?i)\\\\bLIMIT\\\\s+\\\\d+\").matcher(gql).find(); }","typeGuard":null,"tryCatchPattern":"try { q = translateGqlQueryWithLimitCheck(gql, datastore, projectId, db, ns, readTime); } catch (DatastoreException e) { if (e.getCode() != Code.INVALID_ARGUMENT) throw e; /* connector already retried without zero limit */ }","preventionTips":["Omit explicit LIMIT in GQL handed to DatastoreIO so limit probing works.","Prefer structured Query objects over GQL for programmatic reads.","Validate GQL syntax with the Datastore console before pipeline deployment."],"tags":["java","apache-beam","datastore","gql"],"backgroundTag":"invalid-argument-value","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}