{"record":{"id":"8f7c3f35b9359d7e","repo":"apache/beam","slug":"datastore-statistics-for-kind-kind-unavailable","errorCode":null,"errorMessage":"Datastore statistics for kind {kind} unavailable","messagePattern":"Datastore statistics for kind (.+?) unavailable","errorType":"exception","errorClass":"NoSuchElementException","httpStatus":null,"severity":"error","filePath":"sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/datastore/DatastoreV1.java","lineNumber":471,"sourceCode":"      } else {\n        queryBuilder.addKindBuilder().setName(\"__Stat_Ns_Kind__\");\n      }\n\n      queryBuilder.setFilter(\n          makeAndFilter(\n              makeFilter(\"kind_name\", EQUAL, makeValue(ourKind).build()).build(),\n              makeFilter(\"timestamp\", EQUAL, makeValue(latestTimestamp).build()).build()));\n\n      RunQueryRequest request =\n          makeRequest(projectId, databaseId, queryBuilder.build(), namespace, readTime);\n\n      long now = System.currentTimeMillis();\n      RunQueryResponse response = datastore.runQuery(request);\n      LOG.debug(\"Query for per-kind statistics took {}ms\", System.currentTimeMillis() - now);\n\n      QueryResultBatch batch = response.getBatch();\n      if (batch.getEntityResultsCount() == 0) {\n        throw new NoSuchElementException(\n            \"Datastore statistics for kind \" + ourKind + \" unavailable\");\n      }\n      return batch.getEntityResults(0).getEntity();\n    }\n\n    /**\n     * Get the estimated size of the data returned by the given query.\n     *\n     * <p>Cloud Datastore provides no way to get a good estimate of how large the result of a query\n     * entity kind being queried, using the __Stat_Kind__ system table, assuming exactly 1 kind is\n     * specified in the query.\n     *\n     * <p>See https://cloud.google.com/datastore/docs/concepts/stats.\n     */\n    static long getEstimatedSizeBytes(\n        Datastore datastore,\n        String projectId,\n        String databaseId,","sourceCodeStart":453,"sourceCodeEnd":489,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/datastore/DatastoreV1.java#L453-L489","documentation":"getLatestTableStats queries Datastore's per-kind statistics entity (e.g. __Stat_Kind__-derived rows) for a specific kind and returns the first result. If the query succeeds but yields zero entity results, no statistics entry exists for that kind yet, and a NoSuchElementException is thrown. This typically happens because Datastore only generates kind statistics periodically for kinds that contain data.","triggerScenarios":"Calling the entity() sink path that resolves per-kind table stats for a kind whose __Stat_Kind__ query returns no rows — new/empty kind, wrong namespace, or statistics not yet baked at the requested readTime.","commonSituations":"Writing to a newly created Datastore kind before statistics exist; namespace mismatch (kind exists in another namespace); running soon after first writes within the ~24h statistics refresh window; readTime snapshot older than any statistics.","solutions":["Ensure the kind has entities and wait up to 24 hours for Datastore statistics generation, then retry","Check that the namespace parameter matches the namespace containing the kind's entities","Catch NoSuchElementException around getLatestTableStats and skip/defer size-based split logic (treat stats as unavailable)","Retry the query later rather than immediately after first writes to the kind"],"exampleFix":"// before\nEntity stats = getLatestTableStats(projectId, kind, namespace, null);\n// after\nEntity stats;\ntry {\n  stats = getLatestTableStats(projectId, kind, namespace, null);\n} catch (NoSuchElementException e) {\n  LOG.warn(\"No Datastore stats for kind {}; proceeding without size estimates\", kind);\n  stats = null;\n}","handlingStrategy":"try-catch","validationCode":"// Verify the kind has entities before requesting stats:\nQuery q = Query.newGqlQueryBuilder(Query.ResultType.ENTITY,\n    \"SELECT * FROM __Stat_Kind__ WHERE __key__ HAS ANCESTOR KEY('__Stat_Kind__', '\" + kind + \"')\")\n    .setLimit(1).build();\nboolean kindStatsExist = datastore.runQuery(q).getBatch().getEntityResultsCount() > 0;","typeGuard":null,"tryCatchPattern":"Entity stats;\ntry {\n  stats = getLatestTableStats(projectId, kind, namespace, readTime);\n} catch (NoSuchElementException e) {\n  LOG.warn(\"Datastore stats unavailable for kind {}\", kind, e);\n  stats = null;\n}","preventionTips":["Ensure the kind contains entities before expecting statistics","Double-check namespace parameters against your Datastore console","Add a retry/backoff around statistics lookups for freshly written kinds"],"tags":["gcp","datastore","statistics","entity-not-found","bigquery-io"],"backgroundTag":"entity-not-found","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}