{"record":{"id":"e6108ad32a99c6d6","repo":"apache/beam","slug":"error-while-converting-table-schema-s-to-json","errorCode":null,"errorMessage":"Error while converting table schema %s to JSON!","messagePattern":"Error while converting table schema (.+?) to JSON!","errorType":"exception","errorClass":"RuntimeException","httpStatus":null,"severity":"error","filePath":"sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryIO.java","lineNumber":806,"sourceCode":"   *   }\n   * }).from(\"...\");\n   * }</pre>\n   */\n  public static <T> TypedRead<T> read(SerializableFunction<SchemaAndRecord, T> parseFn) {\n    return new AutoValue_BigQueryIO_TypedRead.Builder<T>()\n        .setValidate(true)\n        .setWithTemplateCompatibility(false)\n        .setBigQueryServices(new BigQueryServicesImpl())\n        .setDatumReaderFactory(\n            (SerializableFunction<TableSchema, AvroSource.DatumReaderFactory<T>>)\n                input -> {\n                  try {\n                    String jsonTableSchema = BigQueryIO.JSON_FACTORY.toString(input);\n                    return (AvroSource.DatumReaderFactory<T>)\n                        (writer, reader) ->\n                            new GenericDatumTransformer<>(parseFn, jsonTableSchema, writer);\n                  } catch (IOException e) {\n                    throw new RuntimeException(\n                        \"Error while converting table schema \" + input + \" to JSON!\", e);\n                  }\n                })\n        // TODO: Remove setParseFn once https://github.com/apache/beam/issues/21076 is fixed.\n        .setParseFn(parseFn)\n        .setMethod(TypedRead.Method.DEFAULT)\n        .setUseAvroLogicalTypes(false)\n        .setFormat(DataFormat.AVRO)\n        .setProjectionPushdownApplied(false)\n        .setBadRecordErrorHandler(new DefaultErrorHandler<>())\n        .setBadRecordRouter(BadRecordRouter.THROWING_ROUTER)\n        .build();\n  }\n\n  /**\n   * Reads from a BigQuery table or query and returns a {@link PCollection} with one element per\n   * each row of the table or query result. This API directly deserializes BigQuery AVRO data to the\n   * input class, based on the appropriate {@link org.apache.avro.io.DatumReader}.","sourceCodeStart":788,"sourceCodeEnd":824,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryIO.java#L788-L824","documentation":"Thrown when serializing the BigQuery table schema to JSON (via Jackson JSON_FACTORY) fails with an IOException inside the Avro datum-reader setup for typed reads. Beam needs the schema as JSON to configure GenericDatumTransformer for converting Avro records.","triggerScenarios":"BigQueryIO.read(...).withParseFn(...) using the DEFAULT/FILE_LOADS-style Avro path where the TableSchema cannot be serialized to a JSON string.","commonSituations":"Corrupt or unsupported schema fields coming from BigQuery; Jackson serialization issues in shaded/dependency-conflict environments.","solutions":["Inspect the table schema for unusual/unsupported field types and simplify if possible","Resolve Jackson dependency conflicts (check the shaded beam-sdks-java-io-google-cloud-platform jar is used)","Update to a newer Beam version where schema serialization edge cases are fixed"],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"try {\n  String json = BigQueryIO.JSON_FACTORY.toString(tableSchema);\n} catch (IOException e) {\n  throw new IllegalStateException(\"Failed to serialize table schema to JSON: \" + tableSchema, e);\n}","preventionTips":["Keep Jackson versions aligned with Beam's shading rules","Inspect exotic schema field types before large pipelines","Upgrade Beam when schema-serialization bugs are fixed upstream"],"tags":["bigquery","schema","json","java"],"backgroundTag":"schema-validation-failed","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}