{"record":{"id":"91f39c0cb5adf7f7","repo":"apache/beam","slug":"failure-getting-proto-descriptor-successful-output-will-not","errorCode":null,"errorMessage":"Failure getting proto descriptor. Successful output will not be produced.","messagePattern":"Failure getting proto descriptor\\. Successful output will not be produced\\.","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/StorageApiWriteUnshardedRecords.java","lineNumber":895,"sourceCode":"              int numRecordsAppended = c.protoRows.getSerializedRowsCount();\n              recordsAppended.inc(numRecordsAppended);\n              BigQuerySinkMetrics.appendRowsRowStatusCounter(\n                      BigQuerySinkMetrics.RowStatus.SUCCESSFUL,\n                      BigQuerySinkMetrics.OK,\n                      shortTableUrn)\n                  .inc(numRecordsAppended);\n\n              BigQuerySinkMetrics.reportSuccessfulRpcMetrics(\n                  c, BigQuerySinkMetrics.RpcMethod.APPEND_ROWS, shortTableUrn);\n\n              if (successfulRowsReceiver != null) {\n                Descriptor descriptor = null;\n                try {\n                  descriptor =\n                      TableRowToStorageApiProto.wrapDescriptorProto(\n                          Preconditions.checkStateNotNull(appendClientInfo).getDescriptor());\n                } catch (DescriptorValidationException e) {\n                  LOG.warn(\n                      \"Failure getting proto descriptor. Successful output will not be produced.\",\n                      e);\n                }\n                if (descriptor != null) {\n                  for (int i = 0; i < c.protoRows.getSerializedRowsCount(); ++i) {\n                    ByteString rowBytes = c.protoRows.getSerializedRowsList().get(i);\n                    try {\n                      TableRow row =\n                          TableRowToStorageApiProto.tableRowFromMessage(\n                              Preconditions.checkStateNotNull(appendClientInfo)\n                                  .getSchemaInformation(),\n                              DynamicMessage.parseFrom(descriptor, rowBytes),\n                              true,\n                              successfulRowsPredicate);\n                      org.joda.time.Instant timestamp = c.timestamps.get(i);\n                      successfulRowsReceiver.outputWithTimestamp(row, timestamp);\n                    } catch (Exception e) {\n                      LOG.warn(\"Failure parsing TableRow\", e);","sourceCodeStart":877,"sourceCodeEnd":913,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/StorageApiWriteUnshardedRecords.java#L877-L913","documentation":"In StorageApiWriteUnshardedRecords.flush, when finalizing a batch of rows written via the Storage Write API, Beam converts the proto descriptor to a runtime Descriptor to build failed-row output. If descriptor validation fails (DescriptorValidationException), it logs this warning and skips producing successful/failed row output for that batch.","triggerScenarios":"flush() calls TableRowToStorageApiProto.wrapDescriptorProto on the TableSchemaProto descriptor and validation fails — typically a malformed or inconsistent TableSchema (invalid field names/types) for a Storage API write with error/warning output configured.","commonSituations":"Schemas with field names that violate proto naming rules (invalid identifiers for dynamic protos); unsupported BigQuery types in dynamic destinations; schema updated mid-stream causing mismatched descriptor proto.","solutions":["Inspect the chained DescriptorValidationException to find the offending field and fix the TableSchema (rename invalid fields, use supported types).","Verify the schema is fetched fresh via the Storage API (use storage write's default-schema mode if possible) to avoid stale descriptors.","Avoid characters in BigQuery column names that are invalid in protobuf identifiers.","Restart the pipeline after correcting the schema so a valid descriptor is loaded."],"exampleFix":"// before\n// BigQuery column name: \"my-column\" (hyphen invalid in proto)\n// after\n// BigQuery column name: \"my_column\" (valid proto identifier)","handlingStrategy":"validation","validationCode":"// Validate schema field names are valid proto identifiers before writing:\nboolean protoSafe(String name) { return name != null && name.matches(\"[A-Za-z_][A-Za-z0-9_]*\"); }","typeGuard":null,"tryCatchPattern":"try { descriptor = TableRowToStorageApiProto.wrapDescriptorProto(proto); } catch (DescriptorValidationException e) { LOG.warn(\"Failure getting proto descriptor...\", e); /* route batch to failed-row output */ }","preventionTips":["Use BigQuery column names that are valid protobuf identifiers.","Keep the TableSchema stable during a streaming write; fetch it via the Storage API.","Read the chained DescriptorValidationException to pinpoint bad fields."],"tags":["bigquery","storage-api","protobuf","schema"],"backgroundTag":"schema-validation-failed","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}