{"record":{"id":"6596a7ba4fed531b","repo":"apache/beam","slug":"sending-batchwrite-request-with-writes-totalling-bytes-was","errorCode":null,"errorMessage":"Sending BatchWrite request with {} writes totalling {} bytes was incompletely applied in {}ms ({} ok, {} retryable, {} non-retryable)","messagePattern":"Sending BatchWrite request with (.+?) writes totalling (.+?) bytes was incompletely applied in (.+?)ms \\((.+?) ok, (.+?) retryable, (.+?) non-retryable\\)","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/firestore/FirestoreV1WriteFn.java","lineNumber":486,"sourceCode":"              context,\n              Preconditions.checkArgumentNotNull(okWindow).maxTimestamp(),\n              KV.of(new WriteSuccessSummary(okCount, okBytes), coerceNonNull(okWindow)),\n              () ->\n                  LOG.debug(\n                      \"Sending BatchWrite request with {} writes totalling {} bytes was completely applied in {}ms\",\n                      writesCount,\n                      bytes,\n                      elapsedMillis));\n          attempt.completeSuccess();\n          return DoFlushStatus.OK;\n        } else {\n          if (nonRetryableCount > 0) {\n            int finalOkCount = okCount;\n            handleWriteFailures(\n                context,\n                ImmutableList.copyOf(nonRetryableWrites),\n                () ->\n                    LOG.warn(\n                        \"Sending BatchWrite request with {} writes totalling {} bytes was incompletely applied in {}ms ({} ok, {} retryable, {} non-retryable)\",\n                        writesCount,\n                        bytes,\n                        elapsedMillis,\n                        finalOkCount,\n                        retryableCount,\n                        nonRetryableCount));\n          } else if (retryableCount > 0) {\n            int finalOkCount = okCount;\n            Runnable logMessage =\n                () ->\n                    LOG.debug(\n                        \"Sending BatchWrite request with {} writes totalling {} bytes was incompletely applied in {}ms ({} ok, {} retryable)\",\n                        writesCount,\n                        bytes,\n                        elapsedMillis,\n                        finalOkCount,\n                        retryableCount);","sourceCodeStart":468,"sourceCodeEnd":504,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/firestore/FirestoreV1WriteFn.java#L468-L504","documentation":"FirestoreV1WriteFn flushes accumulated writes in BatchWrite RPCs. A BatchWrite can be partially applied: some writes succeed, some fail with retryable errors, some with non-retryable errors. When any non-retryable failures exist, this warning reports the overall outcome breakdown and each non-retryable failure is routed through handleWriteFailures (typically to a dead-letter output or failure).","triggerScenarios":"BatchWrite responses containing statuses with non-retryable codes such as FAILED_PRECONDITION, PERMISSION_DENIED, NOT_FOUND, or INVALID_ARGUMENT for individual writes — e.g. writing a document with invalid field paths, wrong project/database, or missing collection ancestors.","commonSituations":"Documents referencing a Firestore database ID that doesn't exist; permission changes mid-run; writes exceeding document/field constraints (e.g. invalid document ID characters, field name with invalid characters); stale realtime references after data was deleted concurrently.","solutions":["Inspect the per-write status messages in the failure output / handleWriteFailures result to identify the failing document and specific gRPC code.","Fix non-retryable data issues (invalid document IDs, field names, missing parent collections) at the source before writing.","Verify project ID and databaseId passed to FirestoreIO.write() point to an existing database with correct IAM permissions.","Route failures to a dead-letter output (withFailureHandling or batchWrite path's dead letter PCollection) instead of letting them abort, then reprocess corrected records."],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":"// Java: pre-validate document paths before writing\nif (!docPath.matches(\"projects/[^/]+/databases/[^/]+/documents/([A-Za-z0-9_-]+/[A-Za-z0-9_-]+)+\")) {\n  throw new IllegalArgumentException(\"Invalid Firestore document path: \" + docPath);\n}","typeGuard":null,"tryCatchPattern":"// Route non-retryable failures to dead letter instead of throwing\nPCollection<FirestoreV1.WriteFailure> failures =\n    writes.apply(FirestoreIO.v1().write().batchWrite()).getFailures();\nfailures.apply(ParDo.of(new DoFn<FirestoreV1.WriteFailure, Void>() {\n  @ProcessElement\n  public void process(ProcessContext c) {\n    FirestoreV1.WriteFailure f = c.element();\n    LOG.error(\"write failed code={} msg={}\", f.getStatus().getCode(), f.getStatus().getErrorMessage());\n  }\n}));","preventionTips":["Monitor the getFailures() PCollection from FirestoreIO batch writes.","Validate document IDs (no '/', size limits) and field names before writing.","Pin correct projectId/databaseId in FirestoreIO.v1().write().withProjectId/withDatabaseId.","Alert on non-retryable gRPC codes (PERMISSION_DENIED, FAILED_PRECONDITION, NOT_FOUND)."],"tags":["gcp-firestore","batch-write","partial-failure","beam-io"],"backgroundTag":"api-error-response","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}