{"record":{"id":"733f95c15f56ceac","repo":"apache/beam","slug":"datastore-total-statistics-unavailable","errorCode":null,"errorMessage":"Datastore total statistics unavailable","messagePattern":"Datastore total statistics unavailable","errorType":"exception","errorClass":"NoSuchElementException","httpStatus":null,"severity":"error","filePath":"sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/datastore/DatastoreV1.java","lineNumber":428,"sourceCode":"        @Nullable Instant readTime)\n        throws DatastoreException {\n      Query.Builder query = Query.newBuilder();\n      // Note: namespace either being null or empty represents the default namespace, in which\n      // case we treat it as not provided by the user.\n      if (Strings.isNullOrEmpty(namespace)) {\n        query.addKindBuilder().setName(\"__Stat_Total__\");\n      } else {\n        query.addKindBuilder().setName(\"__Stat_Ns_Total__\");\n      }\n      query.addOrder(makeOrder(\"timestamp\", DESCENDING));\n      query.setLimit(Int32Value.newBuilder().setValue(1));\n      RunQueryRequest request =\n          makeRequest(projectId, databaseId, query.build(), namespace, readTime);\n\n      RunQueryResponse response = datastore.runQuery(request);\n      QueryResultBatch batch = response.getBatch();\n      if (batch.getEntityResultsCount() == 0) {\n        throw new NoSuchElementException(\"Datastore total statistics unavailable\");\n      }\n      Entity entity = batch.getEntityResults(0).getEntity();\n      return entity.getPropertiesOrThrow(\"timestamp\").getTimestampValue().getSeconds() * 1000000;\n    }\n\n    /**\n     * Retrieve latest table statistics for a given kind, namespace, and datastore. If the Read has\n     * readTime specified, the latest statistics at or before readTime is retrieved.\n     */\n    private static Entity getLatestTableStats(\n        String projectId,\n        String databaseId,\n        String ourKind,\n        @Nullable String namespace,\n        Datastore datastore,\n        @Nullable Instant readTime)\n        throws DatastoreException {\n      long latestTimestamp =","sourceCodeStart":410,"sourceCodeEnd":446,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/datastore/DatastoreV1.java#L410-L446","documentation":"DatastoreV1 queries Datastore's built-in __Stat_Total__ statistics entity to learn the latest statistics timestamp. This error means the total-statistics query ran successfully but Datastore returned zero entity results — no total statistics entry exists yet. It surfaces as a NoSuchElementException from queryLatestStatisticsTimestamp, propagating to callers that use the timestamp to pick the freshest per-kind statistics.","triggerScenarios":"Calling latestTimestamp() (via queryLatestStatisticsTimestamp) on a project/database whose __Stat_Total__ query (optionally filtered by namespace and readTime) returns no entity results — i.e. Datastore has not yet produced total statistics, or the namespace has no statistics entities.","commonSituations":"Running against a brand-new project where the Datastore statistics baker has not run yet; querying a namespace with no data; a custom/read-time snapshot predating any statistics; mistyped namespace causing an empty result.","solutions":["Wait 24-48 hours after project/data creation for Datastore to bake statistics, then retry","Verify the namespace argument matches where your entities actually live (or pass null for the default namespace)","Remove or widen the readTime/namespace restriction so the statistics query can match the default total statistics","Catch NoSuchElementException in latestTimestamp and fall back to System.currentTimeMillis() or per-kind statistics without a timestamp"],"exampleFix":"// before\nlong timestamp = latestTimestamp(projectId, namespace, null);\n// after\nlong timestamp;\ntry {\n  timestamp = latestTimestamp(projectId, namespace, null);\n} catch (NoSuchElementException e) {\n  timestamp = System.currentTimeMillis();\n}","handlingStrategy":"try-catch","validationCode":"// Pre-check via a throwaway Datastore query before running the pipeline:\nQuery q = Query.newGqlQueryBuilder(Query.ResultType.ENTITY,\n    \"SELECT * FROM __Stat_Total__\").setLimit(1).build();\nRunQueryResponse r = datastore.runQuery(makeRequest(projectId, namespace, q, null));\nboolean statsAvailable = r.getBatch().getEntityResultsCount() > 0;\nif (!statsAvailable) throw new SkipStatisticsException(\"Datastore total stats not baked yet\");","typeGuard":null,"tryCatchPattern":"long ts;\ntry {\n  ts = latestTimestamp(projectId, namespace, readTime);\n} catch (NoSuchElementException e) {\n  ts = System.currentTimeMillis();\n}","preventionTips":["Wait ~24-48h after creating a project/kind before relying on Datastore statistics","Always catch NoSuchElementException around statistics queries — empty results are expected for new projects","Verify namespace spelling; a wrong namespace silently returns zero statistics rows"],"tags":["gcp","datastore","statistics","empty-result","bigquery-io"],"backgroundTag":"empty-result-set","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}