apache/beam · warning

Sending BatchWrite request with

Error message

Sending BatchWrite request with {} writes totalling {} bytes failed due to error: {}

What it means

A Firestore BatchWrite RPC (sent from FirestoreV1WriteFn during a flush) failed with a RuntimeException. The code logs the failure, records the attempt as failed, and records 0 successful writes so the bundle write logic can retry or dead-letter the writes per the configured retry budget.

Solutions

1. Check per-entry write statuses in the BatchWrite response — BatchWrite returns per-document errors rather than failing the whole RPC; fix data-level issues (size limits, missing documents)
2. Reduce write throughput or batch size to stay under Firestore quotas
3. Verify the service account has cloud datastore user permissions on the target database
4. Ensure retry budget/retry predicate in FirestoreV1 is tuned; transient gRPC errors are retried automatically

Example fix

// before
LOG.warn("Sending BatchWrite request with {} writes totalling {} bytes failed due to error: {}", writesCount, bytes, exceptionMessage != null ? exceptionMessage : exception.getClass().getName());
// after
LOG.warn("BatchWrite failed: {} writes / {} bytes / status={}", writesCount, bytes, exception.getClass().getSimpleName(), exception);

// before flushing, guard against obviously oversized writes
if (bytes > MAX_BATCH_BYTES || writesCount > MAX_BATCH_DOCS) {
  writes = splitBatch(writes);
}

try {
  response = firestoreStub.batchWriteCallable().call(request);
} catch (StatusRuntimeException e) {
  if (RETRYABLE_STATUSES.contains(e.getStatus().getCode())) {
    retryWithBackoff(request);
  } else {
    deadLetter(writes, e);
  }
}

When it happens

Trigger: firestoreStub.batchWriteCallable().call(request) throws at flush time: Firestore quota exceeded, deadline exceeded, document size limits, permission denied, or transient gRPC UNAVAILABLE errors during doFlush (invoked from flushStatus).

Common situations: Pipelines writing at a rate above Firestore write quotas, Firestore security/IAM misconfiguration, long GC pauses causing gRPC deadlines to expire, network instability between the Dataflow worker and Firestore.

Understand the failure class

Background: "API request failed": what wrapped HTTP errors from external APIs mean and how to find the real cause — this error's family across 29 libraries.

Related errors

• Append to stream failed with invalid offset of
• Append to stream failed with Status Code . The stream may…
• could not close data operations client
• Error checking whether table
• Firestore project id must be set on the transform or…

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/8a64581cfafcaf23. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/firestore/FirestoreV1WriteFn.java:413

      //    If an error is encountered and is not retryable, the error will be thrown and the loop
      // will end
      //    If no error is encountered the responses WriteResults will be inspected before breaking
      // the loop
      while (true) {
        Instant start = clock.instant();
        LOG.debug(
            "Sending BatchWrite request with {} writes totalling {} bytes", writesCount, bytes);
        Instant end;
        BatchWriteResponse response;
        try {
          attempt.recordRequestStart(start, writesCount);
          response = firestoreStub.batchWriteCallable().call(request);
          end = clock.instant();
          attempt.recordRequestSuccessful(end);
        } catch (RuntimeException exception) {
          end = clock.instant();
          String exceptionMessage = exception.getMessage();
          LOG.warn(
              "Sending BatchWrite request with {} writes totalling {} bytes failed due to error: {}",
              writesCount,
              bytes,
              exceptionMessage != null ? exceptionMessage : exception.getClass().getName());
          attempt.recordRequestFailed(end);
          attempt.recordWriteCounts(end, 0, writesCount);
          flushBuffer.forEach(writes::offer);
          attempt.checkCanRetry(end, exception);
          continue;
        }

        long elapsedMillis = end.minus(Duration.millis(start.getMillis())).getMillis();

        int okCount = 0;
        long okBytes = 0L;
        BoundedWindow okWindow = null;
        List<KV<WriteFailure, BoundedWindow>> nonRetryableWrites = new ArrayList<>();

View on GitHub (pinned to 12126d8942)