{"record":{"id":"0c02f0b22373390b","repo":"apache/beam","slug":"problem-converting-field-s-expected-type-s","errorCode":null,"errorMessage":"Problem converting field %s expected type: %s","messagePattern":"Problem converting field (.+?) expected type: (.+?)","errorType":"validation","errorClass":"SchemaDoesntMatchException","httpStatus":null,"severity":"error","filePath":"sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/TableRowToStorageApiProto.java","lineNumber":1006,"sourceCode":"          Preconditions.checkArgumentNotNull(builder).setField(fieldDescriptor, value);\n        }\n        // For STRUCT fields, we add a placeholder to unknownFields using the getNestedUnknown\n        // supplier (in case we encounter unknown nested fields). If the placeholder comes out\n        // to be empty, we should clean it up\n        if ((fieldSchemaInformation.getType().equals(TableFieldSchema.Type.STRUCT)\n                && unknownFields != null)\n            && ((unknownFields.get(key) instanceof Map\n                    && ((Map<?, ?>) unknownFields.get(key)).isEmpty()) // single struct, empty\n                || (unknownFields.get(key)\n                        instanceof List // repeated struct, empty list or list with empty structs\n                    && (((List<?>) unknownFields.get(key)).isEmpty()\n                        || ((List<?>) unknownFields.get(key))\n                            .stream()\n                                .allMatch(row -> row == null || ((Map<?, ?>) row).isEmpty()))))) {\n          unknownFields.remove(key);\n        }\n      } catch (Exception e) {\n        throw new SchemaDoesntMatchException(\n            \"Problem converting field \"\n                + fieldSchemaInformation.getFullName()\n                + \" expected type: \"\n                + fieldSchemaInformation.getType(),\n            e);\n      }\n    }\n    if (changeType != null) {\n      Preconditions.checkArgumentNotNull(builder)\n          .setField(\n              Preconditions.checkStateNotNull(\n                  Preconditions.checkArgumentNotNull(descriptor)\n                      .findFieldByName(StorageApiCDC.CHANGE_TYPE_COLUMN)),\n              changeType);\n      Preconditions.checkArgumentNotNull(builder)\n          .setField(\n              Preconditions.checkStateNotNull(\n                  Preconditions.checkArgumentNotNull(descriptor)","sourceCodeStart":988,"sourceCodeEnd":1024,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/TableRowToStorageApiProto.java#L988-L1024","documentation":"During field conversion, any exception while converting a TableRow value to the schema-declared type is caught and rethrown as SchemaDoesntMatchException(\"Problem converting field <name> expected type: <type>\", cause). It signals the row's value cannot be coerced into the declared BigQuery field type (e.g. string into INT64, wrong nested record shape).","triggerScenarios":"Writing a TableRow via the Storage API where a field's runtime value type doesn't match the schema type: non-numeric string for an INTEGER field, malformed timestamp, list where a scalar is required, nested record with wrong sub-shape.","commonSituations":"Upstream data quality issues (nulls/strings in numeric columns), schema changed to a stricter type, timestamps not in expected format, JSON-sourced rows with heterogeneous types.","solutions":["Read the wrapped cause 'e' — it names the exact conversion failure; fix the value format at the source.","Validate/coerce values against the schema (e.g. parse numerics, normalize timestamps to ISO-8601) before writing.","Correct the table schema if the new value type is intentional (e.g. widen INT64 to NUMERIC/STRING).","Catch SchemaDoesntMatchException in the write's error handling and route bad records to a dead-letter sink."],"exampleFix":"// before\nrow.set(\"count\", \"12abc\"); // schema INT64 -> SchemaDoesntMatchException\n// after\nrow.set(\"count\", Long.parseLong(countStr.trim())); // or validate upstream","handlingStrategy":"try-catch","validationCode":"// Pre-write coercion against schema\nObject v = row.get(name);\nif (\"INTEGER\".equals(fieldType) && v instanceof String) {\n  row.set(name, Long.parseLong((String) v));\n}","typeGuard":null,"tryCatchPattern":"try {\n  writeResult = rows.apply(\"WriteBQ\", BigQueryIO.writeTableRows()...);\n} catch (Exception e) {\n  Throwable root = e;\n  while (root.getCause() != null) root = root.getCause(); // find conversion cause\n  LOG.error(\"Field type mismatch: {}\", root.getMessage());\n  throw e;\n}","preventionTips":["Coerce and validate row values against the schema before writing.","Route SchemaDoesntMatchException-failing records to a dead-letter pipeline.","Inspect the wrapped cause for the precise field and conversion problem."],"tags":["java","beam","bigquery","schema","type-mismatch","storage-api"],"backgroundTag":"type-mismatch","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}