{"record":{"id":"94c4d818d50da8fa","repo":"apache/beam","slug":"error-parsing-json","errorCode":null,"errorMessage":"Error parsing JSON","messagePattern":"Error parsing JSON","errorType":"exception","errorClass":"RuntimeException","httpStatus":null,"severity":"error","filePath":"sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/meta/provider/text/TextTableProvider.java","lineNumber":199,"sourceCode":"    public static JsonToRow create(Schema schema) {\n      return create(schema, null);\n    }\n\n    @Override\n    public PCollection<Row> expand(PCollection<String> input) {\n      PCollectionTuple rows =\n          input.apply(\n              ParDo.of(\n                      new DoFn<String, Row>() {\n                        @ProcessElement\n                        public void processElement(ProcessContext context) {\n                          try {\n                            context.output(jsonToRow(getObjectMapper(), context.element()));\n                          } catch (UnsupportedRowJsonException jsonException) {\n                            if (deadLetterFile() != null) {\n                              context.output(DLF_TAG, context.element());\n                            } else {\n                              throw new RuntimeException(\"Error parsing JSON\", jsonException);\n                            }\n                          }\n                        }\n                      })\n                  .withOutputTags(\n                      MAIN_TAG,\n                      deadLetterFile() != null ? TupleTagList.of(DLF_TAG) : TupleTagList.empty()));\n\n      if (deadLetterFile() != null) {\n        rows.get(DLF_TAG).setCoder(StringUtf8Coder.of()).apply(writeJsonToDlf());\n      }\n      return rows.get(MAIN_TAG).setRowSchema(schema());\n    }\n\n    private TextIO.Write writeJsonToDlf() {\n      return TextIO.write().withDelimiter(new char[] {}).to(deadLetterFile());\n    }\n","sourceCodeStart":181,"sourceCodeEnd":217,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/meta/provider/text/TextTableProvider.java#L181-L217","documentation":"During a 'json'-format text table read, each line is converted with jsonToRow. If a line cannot be mapped to the table schema (UnsupportedRowJsonException) and no dead-letter file is configured, the DoFn wraps the cause in a RuntimeException(\"Error parsing JSON\") which fails the pipeline. It exists to surface malformed records instead of silently dropping them.","triggerScenarios":"Reading a text table with format 'json' where a line is malformed JSON, a JSON array instead of object, fields with mismatched types, missing required columns, or unexpected extra structure — with no deadLetterFile configured.","commonSituations":"Mixed plain-text and JSON files in the input glob; single-quoted/JSON5 lines; schema changed after files were written; nested fields not matching the declared Beam schema.","solutions":["Configure a dead-letter file (deadLetterFile) so bad records are routed to DLF_TAG instead of failing the pipeline.","Fix the offending JSON records so they conform to the declared schema.","Align the table schema with the actual JSON structure (types, required fields).","Pre-clean/validate input files before running the pipeline."],"exampleFix":"// before: no dead letter -> pipeline fails on first bad record\n// after: route bad records to a dead-letter output instead of throwing\nTextTableProvider.JsonToRowFn fn =\n    new TextTableProvider.JsonToRowFn(schema, \"gs://bucket/dlq/errors\");","handlingStrategy":"fallback","validationCode":"// validate each line parses before pipeline run (offline precheck)\nnew ObjectMapper().readValue(line, Map.class);","typeGuard":null,"tryCatchPattern":"try {\n  pipelineResult.waitUntilFinish();\n} catch (RuntimeException e) {\n  if (e.getMessage() != null && e.getMessage().contains(\"Error parsing JSON\")) {\n    // inspect dead-letter / input files for malformed records\n  }\n}","preventionTips":["Always configure a deadLetterFile for json-format text tables.","Validate schema alignment with a sample file before full runs.","Exclude non-JSON files from the input glob pattern."],"tags":["beam-sql","json","record-parsing"],"backgroundTag":"json-parse-error","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}