{"record":{"id":"1d0ea02af42bc430","repo":"apache/beam","slug":"inconsistent-types-seen-for-field-e-getmissingfield-oldvalue","errorCode":null,"errorMessage":"Inconsistent types seen for field: ${e.getMissingField()} ${oldValue.getType()} v.s. ${type}","messagePattern":"Inconsistent types seen for field: (.+?) (.+?) v\\.s\\. (.+?)","errorType":"exception","errorClass":"SchemaDoesntMatchException","httpStatus":null,"severity":"error","filePath":"sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/UpgradeTableSchema.java","lineNumber":93,"sourceCode":"            e.isRepeated() ? TableFieldSchema.Mode.REPEATED : TableFieldSchema.Mode.NULLABLE;\n        // TODO(reuvenlax): Fix this so that arbitrary types can be selected.\n        TableFieldSchema.Type type =\n            e.isStruct() ? TableFieldSchema.Type.STRUCT : TableFieldSchema.Type.STRING;\n        @Nullable TableFieldSchema oldValue =\n            newFields\n                .computeIfAbsent(prefix, p -> Maps.newLinkedHashMap())\n                .put(\n                    name,\n                    TableFieldSchema.newBuilder()\n                        .setName(name)\n                        .setMode(mode)\n                        .setType(type)\n                        .build());\n        if (oldValue != null) {\n          // Duplicates are ok because we might run this over an entire bundle. However we must\n          // ensure that they are compatible.\n          if (!oldValue.getType().equals(type)) {\n            throw new TableRowToStorageApiProto.SchemaDoesntMatchException(\n                \"Inconsistent types seen for field: \"\n                    + e.getMissingField()\n                    + \" \"\n                    + oldValue.getType()\n                    + \" v.s. \"\n                    + type);\n          }\n        }\n      } else if (schemaConversionException\n          instanceof TableRowToStorageApiProto.SchemaMissingRequiredFieldException) {\n        ((TableRowToStorageApiProto.SchemaMissingRequiredFieldException) schemaConversionException)\n            .getMissingFields()\n            .forEach(\n                f -> {\n                  List<String> components = Arrays.asList(f.toLowerCase().split(\"\\\\.\"));\n                  String prefix = String.join(\".\", components.subList(0, components.size() - 1));\n                  String name = components.get(components.size() - 1);\n                  relaxedFields.computeIfAbsent(prefix, p -> Sets.newHashSet()).add(name);","sourceCodeStart":75,"sourceCodeEnd":111,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/UpgradeTableSchema.java#L75-L111","documentation":"UpgradeTableSchema.getIncrementalSchema merges field schemas encountered across elements into a single incremental schema. Duplicate fields are tolerated, but if the same field is seen with two different types it throws SchemaDoesntMatchException describing the field and the conflicting types, since BigQuery schema evolution cannot reconcile them.","triggerScenarios":"Processing elements where the same missing field is filled with different BigQuery types (e.g. STRING vs INTEGER) — often because values vary in Java type across bundle elements, causing type inference to derive different TableFieldSchema types.","commonSituations":"Heterogeneous records in one PCollection (polymorphic rows), a field sometimes null/absent and typed differently when present, numeric values sometimes inferred as INTEGER and sometimes as FLOAT, or JSON-sourced rows with mixed-type columns.","solutions":["Make the field's type consistent across all elements — coerce values to one type before writing","Explicitly declare the field's type in the BigQuery schema so inference can't diverge","Normalize numeric values (e.g. always emit Double or always Long) in your TableRow construction","Split heterogeneous elements into separate PCollections/destinations with their own schemas","Log or quarantine the offending records where e.getMissingField() has inconsistent values"],"exampleFix":"// before\nrow.set(\"amount\", maybeIntValue); // sometimes Long, sometimes Double\n// after\nrow.set(\"amount\", ((Number) maybeIntValue).doubleValue()); // always FLOAT type","handlingStrategy":"validation","validationCode":"// normalize field types before writing\nObject v = row.get(fieldName);\nif (v instanceof Number) row.set(fieldName, ((Number) v).doubleValue()); // force FLOAT consistently","typeGuard":"Object coerceConsistentType(Object v, Class<?> expected) {\n  if (v == null) return null;\n  if (expected == Double.class && v instanceof Number) return ((Number) v).doubleValue();\n  if (expected == Long.class && v instanceof Number) return ((Number) v).longValue();\n  if (expected == String.class) return String.valueOf(v);\n  return v;\n}","tryCatchPattern":"try {\n  upgradedSchema = UpgradeTableSchema.getIncrementalSchema(...);\n} catch (TableRowToStorageApiProto.SchemaDoesntMatchException e) {\n  LOG.error(\"Inconsistent type for field {}: {}\", e.getMissingField(), e.getMessage());\n  // coerce offending field or route record to DLQ\n}","preventionTips":["Coerce numeric values to a single Java type when building TableRows","Declare explicit field types in the BigQuery schema instead of relying on inference","Keep record shapes homogeneous within a destination","Validate mixed-type fields (JSON sources) before writing"],"tags":["java","google-bigquery","schema","type-mismatch"],"backgroundTag":"schema-validation-failed","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}