apache/beam · error · NoSuchElementException

Datastore total statistics unavailable

Error message

Datastore total statistics unavailable

What it means

DatastoreV1 queries Datastore's built-in __Stat_Total__ statistics entity to learn the latest statistics timestamp. This error means the total-statistics query ran successfully but Datastore returned zero entity results — no total statistics entry exists yet. It surfaces as a NoSuchElementException from queryLatestStatisticsTimestamp, propagating to callers that use the timestamp to pick the freshest per-kind statistics.

Source

Thrown at sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/datastore/DatastoreV1.java:428

        @Nullable Instant readTime)
        throws DatastoreException {
      Query.Builder query = Query.newBuilder();
      // Note: namespace either being null or empty represents the default namespace, in which
      // case we treat it as not provided by the user.
      if (Strings.isNullOrEmpty(namespace)) {
        query.addKindBuilder().setName("__Stat_Total__");
      } else {
        query.addKindBuilder().setName("__Stat_Ns_Total__");
      }
      query.addOrder(makeOrder("timestamp", DESCENDING));
      query.setLimit(Int32Value.newBuilder().setValue(1));
      RunQueryRequest request =
          makeRequest(projectId, databaseId, query.build(), namespace, readTime);

      RunQueryResponse response = datastore.runQuery(request);
      QueryResultBatch batch = response.getBatch();
      if (batch.getEntityResultsCount() == 0) {
        throw new NoSuchElementException("Datastore total statistics unavailable");
      }
      Entity entity = batch.getEntityResults(0).getEntity();
      return entity.getPropertiesOrThrow("timestamp").getTimestampValue().getSeconds() * 1000000;
    }

    /**
     * Retrieve latest table statistics for a given kind, namespace, and datastore. If the Read has
     * readTime specified, the latest statistics at or before readTime is retrieved.
     */
    private static Entity getLatestTableStats(
        String projectId,
        String databaseId,
        String ourKind,
        @Nullable String namespace,
        Datastore datastore,
        @Nullable Instant readTime)
        throws DatastoreException {
      long latestTimestamp =

View on GitHub (pinned to 12126d8942)

Solutions

  1. Wait 24-48 hours after project/data creation for Datastore to bake statistics, then retry
  2. Verify the namespace argument matches where your entities actually live (or pass null for the default namespace)
  3. Remove or widen the readTime/namespace restriction so the statistics query can match the default total statistics
  4. Catch NoSuchElementException in latestTimestamp and fall back to System.currentTimeMillis() or per-kind statistics without a timestamp

Example fix

// before
long timestamp = latestTimestamp(projectId, namespace, null);
// after
long timestamp;
try {
  timestamp = latestTimestamp(projectId, namespace, null);
} catch (NoSuchElementException e) {
  timestamp = System.currentTimeMillis();
}
Defensive patterns

Strategy: try-catch

Validate before calling

// Pre-check via a throwaway Datastore query before running the pipeline:
Query q = Query.newGqlQueryBuilder(Query.ResultType.ENTITY,
    "SELECT * FROM __Stat_Total__").setLimit(1).build();
RunQueryResponse r = datastore.runQuery(makeRequest(projectId, namespace, q, null));
boolean statsAvailable = r.getBatch().getEntityResultsCount() > 0;
if (!statsAvailable) throw new SkipStatisticsException("Datastore total stats not baked yet");

Try / catch

long ts;
try {
  ts = latestTimestamp(projectId, namespace, readTime);
} catch (NoSuchElementException e) {
  ts = System.currentTimeMillis();
}

Prevention

When it happens

Trigger: Calling latestTimestamp() (via queryLatestStatisticsTimestamp) on a project/database whose __Stat_Total__ query (optionally filtered by namespace and readTime) returns no entity results — i.e. Datastore has not yet produced total statistics, or the namespace has no statistics entities.

Common situations: Running against a brand-new project where the Datastore statistics baker has not run yet; querying a namespace with no data; a custom/read-time snapshot predating any statistics; mistyped namespace causing an empty result.

Understand the failure class

Background: EmptyResultError / "no results found": when an API or scraper succeeds but returns zero rows — this error's family across 9 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/733f95c15f56ceac. Report an issue: GitHub.