{"record":{"id":"ac7278e5dfd69373","repo":"apache/beam","slug":"found-json-type-in-tableschema-for-file-loads-write-method-ac7278","errorCode":null,"errorMessage":"Found JSON type in TableSchema for 'FILE_LOADS' write method. \n check steps in https://cloud.google.com/bigquery/docs/loading-data-cloud-storage-avro#extract_json_data_from_avro_data  to ensure the read as a JSON type. Otherwise it will read as a raw (escaped) string.","messagePattern":"Found JSON type in TableSchema for 'FILE_LOADS' write method\\. \n check steps in https://cloud\\.google\\.com/bigquery/docs/loading-data-cloud-storage-avro#extract_json_data_from_avro_data  to ensure the read as a JSON type\\. Otherwise it will read as a raw \\(escaped\\) string\\.","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryIO.java","lineNumber":4245,"sourceCode":"          LOG.warn(\n              \"Error Handling is partially supported when using FILE_LOADS. Consider using STORAGE_WRITE_API or STORAGE_API_AT_LEAST_ONCE\");\n        }\n\n        // Batch load handles wrapped json string value differently than the other methods. Raise a\n        // warning when applies.\n        ValueProvider<String> jsonSchema = getJsonSchema();\n        if (jsonSchema != null && jsonSchema.isAccessible()) {\n          JsonElement schema = JsonParser.parseString(jsonSchema.get());\n          if (!schema.getAsJsonObject().keySet().isEmpty() && hasJsonTypeInSchema(schema)) {\n            if (rowWriterFactory.getOutputType() == OutputType.JsonTableRow) {\n              LOG.warn(\n                  \"Found JSON type in TableSchema for 'FILE_LOADS' write method. \\n\"\n                      + \"Make sure the TableRow value is a Jackson JsonNode to ensure the read as a \"\n                      + \"JSON type. Otherwise it will read as a raw (escaped) string.\\n\"\n                      + \"See https://cloud.google.com/bigquery/docs/loading-data-cloud-storage-json#limitations \"\n                      + \"for limitations.\");\n            } else if (rowWriterFactory.getOutputType() == OutputType.AvroGenericRecord) {\n              LOG.warn(\n                  \"Found JSON type in TableSchema for 'FILE_LOADS' write method. \\n\"\n                      + \" check steps in https://cloud.google.com/bigquery/docs/loading-data-cloud-storage-avro#extract_json_data_from_avro_data \"\n                      + \" to ensure the read as a JSON type. Otherwise it will read as a raw \"\n                      + \"(escaped) string.\");\n            }\n          }\n        }\n\n        BatchLoads<DestinationT, T> batchLoads =\n            new BatchLoads<>(\n                getWriteDisposition(),\n                getCreateDisposition(),\n                getJsonTableRef() != null,\n                dynamicDestinations,\n                destinationCoder,\n                getCustomGcsTempLocation(),\n                getLoadJobProjectId(),\n                getIgnoreUnknownValues(),","sourceCodeStart":4227,"sourceCodeEnd":4263,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryIO.java#L4227-L4263","documentation":"This is a log warning from BigQueryIO for FILE_LOADS writes using Avro output. When the table schema has JSON-typed fields and rows are serialized as AvroGenericRecord, Beam warns that special extraction steps must be followed or the JSON will be read back as raw escaped strings in BigQuery.","triggerScenarios":"Using BigQueryIO write with withMethod(FILE_LOADS) and AvroGenericRecord output type where the destination schema (via withJsonSchema) contains JSON-type fields.","commonSituations":"Pipelines writing Avro records to BigQuery tables with JSON columns; migrating a pipeline from TableRow to Avro serialization without adjusting JSON handling; following BigQuery docs for extracting JSON from Avro data.","solutions":["Follow https://cloud.google.com/bigquery/docs/loading-data-cloud-storage-avro#extract_json_data_from_avro_data to encode JSON fields as strings in the Avro record so BigQuery can extract them.","Store JSON fields as STRING in the Avro schema and let BigQuery's extract step convert them to JSON type.","Switch output type to JsonTableRow with Jackson JsonNode values if Avro handling is too constrained.","Remove JSON types from the table schema (use STRING) if structured JSON access is not required."],"exampleFix":"// before\nrecord.put(\"payload\", schema.getField(\"payload\").schema()); // nested/complex Avro schema\n// after\nrecord.put(\"payload\", \"{\\\"a\\\":1}\"); // JSON encoded as string, per BigQuery Avro extraction docs","handlingStrategy":"validation","validationCode":"// Verify Avro output type when schema has JSON fields\nif (schemaContainsJsonType(jsonSchemaString) && outputType != OutputType.JsonTableRow) {\n  // encode JSON columns as strings per Avro extraction docs before writing\n}","typeGuard":"boolean isGenericRecordOutput(Object writer) { return writer instanceof org.apache.avro.generic.GenericRecord; }","tryCatchPattern":null,"preventionTips":["Follow the BigQuery Avro JSON extraction documented steps when writing Avro.","Avoid JSON column types in schemas unless you control serialization end-to-end.","Test round-trip loads into a scratch dataset to confirm JSON parses as JSON, not escaped strings."],"tags":["bigquery","avro","json","file-loads"],"backgroundTag":"type-mismatch","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}