Sending BatchWrite request with

Error message

Sending BatchWrite request with {} writes totalling {} bytes was incompletely applied in {}ms ({} ok, {} retryable, {} non-retryable)

What it means

FirestoreV1WriteFn flushes accumulated writes in BatchWrite RPCs. A BatchWrite can be partially applied: some writes succeed, some fail with retryable errors, some with non-retryable errors. When any non-retryable failures exist, this warning reports the overall outcome breakdown and each non-retryable failure is routed through handleWriteFailures (typically to a dead-letter output or failure).

Solutions

Inspect the per-write status messages in the failure output / handleWriteFailures result to identify the failing document and specific gRPC code.
Fix non-retryable data issues (invalid document IDs, field names, missing parent collections) at the source before writing.
Verify project ID and databaseId passed to FirestoreIO.write() point to an existing database with correct IAM permissions.
Route failures to a dead-letter output (withFailureHandling or batchWrite path's dead letter PCollection) instead of letting them abort, then reprocess corrected records.

Defensive patterns

Strategy: try-catch

Validate before calling

// Java: pre-validate document paths before writing
if (!docPath.matches("projects/[^/]+/databases/[^/]+/documents/([A-Za-z0-9_-]+/[A-Za-z0-9_-]+)+")) {
  throw new IllegalArgumentException("Invalid Firestore document path: " + docPath);
}

Try / catch

// Route non-retryable failures to dead letter instead of throwing
PCollection<FirestoreV1.WriteFailure> failures =
    writes.apply(FirestoreIO.v1().write().batchWrite()).getFailures();
failures.apply(ParDo.of(new DoFn<FirestoreV1.WriteFailure, Void>() {
  @ProcessElement
  public void process(ProcessContext c) {
    FirestoreV1.WriteFailure f = c.element();
    LOG.error("write failed code={} msg={}", f.getStatus().getCode(), f.getStatus().getErrorMessage());
  }
}));

Prevention

Monitor the getFailures() PCollection from FirestoreIO batch writes.
Validate document IDs (no '/', size limits) and field names before writing.
Pin correct projectId/databaseId in FirestoreIO.v1().write().withProjectId/withDatabaseId.
Alert on non-retryable gRPC codes (PERMISSION_DENIED, FAILED_PRECONDITION, NOT_FOUND).

When it happens

Trigger: BatchWrite responses containing statuses with non-retryable codes such as FAILED_PRECONDITION, PERMISSION_DENIED, NOT_FOUND, or INVALID_ARGUMENT for individual writes — e.g. writing a document with invalid field paths, wrong project/database, or missing collection ancestors.

Common situations: Documents referencing a Firestore database ID that doesn't exist; permission changes mid-run; writes exceeding document/field constraints (e.g. invalid document ID characters, field name with invalid characters); stale realtime references after data was deleted concurrently.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/6596a7ba4fed531b. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/firestore/FirestoreV1WriteFn.java:486

              context,
              Preconditions.checkArgumentNotNull(okWindow).maxTimestamp(),
              KV.of(new WriteSuccessSummary(okCount, okBytes), coerceNonNull(okWindow)),
              () ->
                  LOG.debug(
                      "Sending BatchWrite request with {} writes totalling {} bytes was completely applied in {}ms",
                      writesCount,
                      bytes,
                      elapsedMillis));
          attempt.completeSuccess();
          return DoFlushStatus.OK;
        } else {
          if (nonRetryableCount > 0) {
            int finalOkCount = okCount;
            handleWriteFailures(
                context,
                ImmutableList.copyOf(nonRetryableWrites),
                () ->
                    LOG.warn(
                        "Sending BatchWrite request with {} writes totalling {} bytes was incompletely applied in {}ms ({} ok, {} retryable, {} non-retryable)",
                        writesCount,
                        bytes,
                        elapsedMillis,
                        finalOkCount,
                        retryableCount,
                        nonRetryableCount));
          } else if (retryableCount > 0) {
            int finalOkCount = okCount;
            Runnable logMessage =
                () ->
                    LOG.debug(
                        "Sending BatchWrite request with {} writes totalling {} bytes was incompletely applied in {}ms ({} ok, {} retryable)",
                        writesCount,
                        bytes,
                        elapsedMillis,
                        finalOkCount,
                        retryableCount);

View on GitHub (pinned to 12126d8942)