{"record":{"id":"c6aa6d5ec88172da","repo":"apache/beam","slug":"found-json-type-in-tableschema-for-file-loads-write-method","errorCode":null,"errorMessage":"Found JSON type in TableSchema for 'FILE_LOADS' write method. \nMake sure the TableRow value is a Jackson JsonNode to ensure the read as a JSON type. Otherwise it will read as a raw (escaped) string.\nSee https://cloud.google.com/bigquery/docs/loading-data-cloud-storage-json#limitations for limitations.","messagePattern":"Found JSON type in TableSchema for 'FILE_LOADS' write method\\. \nMake sure the TableRow value is a Jackson JsonNode to ensure the read as a JSON type\\. Otherwise it will read as a raw \\(escaped\\) string\\.\nSee https://cloud\\.google\\.com/bigquery/docs/loading-data-cloud-storage-json#limitations for limitations\\.","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryIO.java","lineNumber":4238,"sourceCode":"              rowWriterFactory.getOutputType() == OutputType.AvroGenericRecord,\n              \"useAvroLogicalTypes can only be set with Avro output.\");\n        }\n        checkArgument(\n            !getPropagateSuccessfulStorageApiWrites(),\n            \"withPropagateSuccessfulStorageApiWrites only supported when using storage api writes.\");\n        if (!(getBadRecordRouter() instanceof ThrowingBadRecordRouter)) {\n          LOG.warn(\n              \"Error Handling is partially supported when using FILE_LOADS. Consider using STORAGE_WRITE_API or STORAGE_API_AT_LEAST_ONCE\");\n        }\n\n        // Batch load handles wrapped json string value differently than the other methods. Raise a\n        // warning when applies.\n        ValueProvider<String> jsonSchema = getJsonSchema();\n        if (jsonSchema != null && jsonSchema.isAccessible()) {\n          JsonElement schema = JsonParser.parseString(jsonSchema.get());\n          if (!schema.getAsJsonObject().keySet().isEmpty() && hasJsonTypeInSchema(schema)) {\n            if (rowWriterFactory.getOutputType() == OutputType.JsonTableRow) {\n              LOG.warn(\n                  \"Found JSON type in TableSchema for 'FILE_LOADS' write method. \\n\"\n                      + \"Make sure the TableRow value is a Jackson JsonNode to ensure the read as a \"\n                      + \"JSON type. Otherwise it will read as a raw (escaped) string.\\n\"\n                      + \"See https://cloud.google.com/bigquery/docs/loading-data-cloud-storage-json#limitations \"\n                      + \"for limitations.\");\n            } else if (rowWriterFactory.getOutputType() == OutputType.AvroGenericRecord) {\n              LOG.warn(\n                  \"Found JSON type in TableSchema for 'FILE_LOADS' write method. \\n\"\n                      + \" check steps in https://cloud.google.com/bigquery/docs/loading-data-cloud-storage-avro#extract_json_data_from_avro_data \"\n                      + \" to ensure the read as a JSON type. Otherwise it will read as a raw \"\n                      + \"(escaped) string.\");\n            }\n          }\n        }\n\n        BatchLoads<DestinationT, T> batchLoads =\n            new BatchLoads<>(\n                getWriteDisposition(),","sourceCodeStart":4220,"sourceCodeEnd":4256,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryIO.java#L4220-L4256","documentation":"This is a log warning from Apache Beam's BigQueryIO. When a FILE_LOADS write targets a table whose schema contains JSON-type fields, Beam checks how rows are being serialized. If rows are written as JsonTableRow but the TableRow values are plain strings instead of Jackson JsonNode objects, the data will land in BigQuery as raw escaped strings rather than structured JSON.","triggerScenarios":"Using BigQueryIO.writeTableRows() with withMethod(FILE_LOADS) and withAvroFormat/JsonTableRow output where the destination table schema (from withJsonSchema or inferred) contains a JSON field type, and the TableRow values are plain String instead of Jackson JsonNode.","commonSituations":"Developers loading JSON data into BigQuery tables with JSON-typed columns; specifying a JSON schema via JSON string but supplying string values in TableRow; schema drift where a column changed from STRING to JSON type.","solutions":["Wrap the value in a Jackson JsonNode (e.g. JsonNode node = new ObjectMapper().valueToTree(obj) or TextNode) instead of a plain String in the TableRow.","Verify rowWriterFactory output type is OutputType.JsonTableRow when using JSON schema fields.","Consult the BigQuery JSON loading limitations doc and adjust the schema if JSON type is unnecessary.","Switch the write method (e.g. STORAGE_API_WRITES) if FILE_LOADS JSON handling does not fit the pipeline."],"exampleFix":"// before\nrow.set(\"payload\", \"{\\\"a\\\":1}\");\n// after\nrow.set(\"payload\", new ObjectMapper().valueToTree(java.util.Map.of(\"a\", 1))); // Jackson JsonNode","handlingStrategy":"validation","validationCode":"boolean hasJsonNode(org.apache.beam.sdk.values.Row row, ObjectMapper mapper) {\n  return row.getValues().stream()\n      .noneMatch(v -> v instanceof String s && s.trim().startsWith(\"{\"));\n}\n// Ensure JSON-typed schema fields receive Jackson JsonNode values: validate before writing.","typeGuard":"boolean isJsonNode(Object v) { return v instanceof com.fasterxml.jackson.databind.JsonNode; }","tryCatchPattern":null,"preventionTips":["Always wrap JSON values in TableRow with Jackson JsonNode when the schema declares JSON type.","Keep table schema and row value types in sync; test FILE_LOADS writes against a staging table.","Review BigQuery JSON load limitations before choosing JSON column types."],"tags":["bigquery","json","serialization","file-loads"],"backgroundTag":"type-mismatch","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}