apache/beam · warning

Found JSON type in TableSchema for 'FILE_LOADS' write…

Error message

Found JSON type in TableSchema for 'FILE_LOADS' write method. 
Make sure the TableRow value is a Jackson JsonNode to ensure the read as a JSON type. Otherwise it will read as a raw (escaped) string.
See https://cloud.google.com/bigquery/docs/loading-data-cloud-storage-json#limitations for limitations.

What it means

This is a log warning from Apache Beam's BigQueryIO. When a FILE_LOADS write targets a table whose schema contains JSON-type fields, Beam checks how rows are being serialized. If rows are written as JsonTableRow but the TableRow values are plain strings instead of Jackson JsonNode objects, the data will land in BigQuery as raw escaped strings rather than structured JSON.

Solutions

  1. Wrap the value in a Jackson JsonNode (e.g. JsonNode node = new ObjectMapper().valueToTree(obj) or TextNode) instead of a plain String in the TableRow.
  2. Verify rowWriterFactory output type is OutputType.JsonTableRow when using JSON schema fields.
  3. Consult the BigQuery JSON loading limitations doc and adjust the schema if JSON type is unnecessary.
  4. Switch the write method (e.g. STORAGE_API_WRITES) if FILE_LOADS JSON handling does not fit the pipeline.

Example fix

// before
row.set("payload", "{\"a\":1}");
// after
row.set("payload", new ObjectMapper().valueToTree(java.util.Map.of("a", 1))); // Jackson JsonNode
Defensive patterns

Strategy: validation

Validate before calling

boolean hasJsonNode(org.apache.beam.sdk.values.Row row, ObjectMapper mapper) {
  return row.getValues().stream()
      .noneMatch(v -> v instanceof String s && s.trim().startsWith("{"));
}
// Ensure JSON-typed schema fields receive Jackson JsonNode values: validate before writing.

Type guard

boolean isJsonNode(Object v) { return v instanceof com.fasterxml.jackson.databind.JsonNode; }

Prevention

When it happens

Trigger: Using BigQueryIO.writeTableRows() with withMethod(FILE_LOADS) and withAvroFormat/JsonTableRow output where the destination table schema (from withJsonSchema or inferred) contains a JSON field type, and the TableRow values are plain String instead of Jackson JsonNode.

Common situations: Developers loading JSON data into BigQuery tables with JSON-typed columns; specifying a JSON schema via JSON string but supplying string values in TableRow; schema drift where a column changed from STRING to JSON type.

Understand the failure class

Background: Type mismatch errors: IllegalArgumentException, TypeError and type guards across 150 open-source libraries — this error's family across 150 libraries.

Related errors


AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/c6aa6d5ec88172da. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryIO.java:4238

              rowWriterFactory.getOutputType() == OutputType.AvroGenericRecord,
              "useAvroLogicalTypes can only be set with Avro output.");
        }
        checkArgument(
            !getPropagateSuccessfulStorageApiWrites(),
            "withPropagateSuccessfulStorageApiWrites only supported when using storage api writes.");
        if (!(getBadRecordRouter() instanceof ThrowingBadRecordRouter)) {
          LOG.warn(
              "Error Handling is partially supported when using FILE_LOADS. Consider using STORAGE_WRITE_API or STORAGE_API_AT_LEAST_ONCE");
        }

        // Batch load handles wrapped json string value differently than the other methods. Raise a
        // warning when applies.
        ValueProvider<String> jsonSchema = getJsonSchema();
        if (jsonSchema != null && jsonSchema.isAccessible()) {
          JsonElement schema = JsonParser.parseString(jsonSchema.get());
          if (!schema.getAsJsonObject().keySet().isEmpty() && hasJsonTypeInSchema(schema)) {
            if (rowWriterFactory.getOutputType() == OutputType.JsonTableRow) {
              LOG.warn(
                  "Found JSON type in TableSchema for 'FILE_LOADS' write method. \n"
                      + "Make sure the TableRow value is a Jackson JsonNode to ensure the read as a "
                      + "JSON type. Otherwise it will read as a raw (escaped) string.\n"
                      + "See https://cloud.google.com/bigquery/docs/loading-data-cloud-storage-json#limitations "
                      + "for limitations.");
            } else if (rowWriterFactory.getOutputType() == OutputType.AvroGenericRecord) {
              LOG.warn(
                  "Found JSON type in TableSchema for 'FILE_LOADS' write method. \n"
                      + " check steps in https://cloud.google.com/bigquery/docs/loading-data-cloud-storage-avro#extract_json_data_from_avro_data "
                      + " to ensure the read as a JSON type. Otherwise it will read as a raw "
                      + "(escaped) string.");
            }
          }
        }

        BatchLoads<DestinationT, T> batchLoads =
            new BatchLoads<>(
                getWriteDisposition(),

View on GitHub (pinned to 12126d8942)