{"record":{"id":"70e79df850c382c5","repo":"apache/beam","slug":"error-while-parsing-the-element","errorCode":null,"errorMessage":"Error while parsing the element","messagePattern":"Error while parsing the element","errorType":"console","errorClass":null,"httpStatus":null,"severity":"error","filePath":"sdks/java/core/src/main/java/org/apache/beam/sdk/io/TFRecordReadSchemaTransformProvider.java","lineNumber":187,"sourceCode":"        Schema errorSchema,\n        boolean handleErrors) {\n      this.errorCounter = Metrics.counter(TFRecordReadSchemaTransformProvider.class, name);\n      this.valueMapper = valueMapper;\n      this.handleErrors = handleErrors;\n      this.errorSchema = errorSchema;\n    }\n\n    @ProcessElement\n    public void process(@DoFn.Element byte[] msg, MultiOutputReceiver receiver) {\n      Row mappedRow = null;\n      try {\n        mappedRow = valueMapper.apply(msg);\n      } catch (Exception e) {\n        if (!handleErrors) {\n          throw new RuntimeException(e);\n        }\n        errorsInBundle += 1;\n        LOG.warn(\"Error while parsing the element\", e);\n        receiver.get(ERROR_TAG).output(ErrorHandling.errorRecord(errorSchema, msg, e));\n      }\n      if (mappedRow != null) {\n        receiver.get(OUTPUT_TAG).output(mappedRow);\n      }\n    }\n\n    @FinishBundle\n    public void finish(FinishBundleContext c) {\n      errorCounter.inc(errorsInBundle);\n      errorsInBundle = 0L;\n    }\n  }\n}\n","sourceCodeStart":169,"sourceCodeEnd":202,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/core/src/main/java/org/apache/beam/sdk/io/TFRecordReadSchemaTransformProvider.java#L169-L202","documentation":"TFRecordReadSchemaTransformProvider's DoFn parses each TFRecord byte[] into a Row via a configured valueMapper. If parsing throws and error handling is enabled (handleErrors), the element is logged at WARN and routed to the ERROR_TAG output as an ErrorHandling record instead of failing the pipeline; if error handling is disabled, the exception is rethrown as a RuntimeException and fails the bundle.","triggerScenarios":"Reading TFRecord files with a valueMapper (e.g. protobuf/json parser) that throws on a malformed or incompatible record while the transform is configured with an error output (handleErrors true) — or throwing a RuntimeException when handleErrors is false.","commonSituations":"Corrupt or truncated TFRecord files; schema/proto mismatch after a schema change; wrong parser configured for the record format.","solutions":["Inspect the ErrorHandling error records on the error output PCollection; log/fix the offending records.","Verify the valueMapper/parser matches the actual record encoding in the TFRecord files.","If handleErrors was disabled, enable error handling to divert bad records instead of failing the pipeline.","Regenerate or re-export corrupted source files."],"exampleFix":"// before (no error output configured)\n.write() // throws on bad record\n// after\n.withErrorHandling()  // routes parse failures to the error output instead of crashing","handlingStrategy":"try-catch","validationCode":"// Sanity-check the record parses before running the pipeline on a sample\nbyte[] sample = readFirstRecord(path);\nassertDoesNotThrow(() -> valueMapper.apply(sample));","typeGuard":null,"tryCatchPattern":"try {\n  Row row = valueMapper.apply(bytes);\n} catch (Exception e) {\n  // mirror the transform: route to dead-letter instead of failing\n  errorOutput.emit(ErrorHandling.errorRecord(errorSchema, bytes, e));\n}","preventionTips":["Always configure error handling for reads over potentially corrupt external data.","Validate a sample of the source files against the parser schema before launching.","Monitor the error-output PCollection size and alert on spikes.","Pin schema/proto versions to avoid mismatched parsers."],"tags":["io","tfrecord","parsing","beam","error-handling"],"backgroundTag":"record-parse-failed","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}