{"record":{"id":"755fa473430e5dbc","repo":"apache/beam","slug":"failed-to-parse-payload-s-as-json-at-s-s-dropping-resource","errorCode":null,"errorMessage":"Failed to parse payload: %s as json at: %s : %s.Dropping resource from batch import.","messagePattern":"Failed to parse payload: (.+?) as json at: (.+?) : (.+?)\\.Dropping resource from batch import\\.","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/healthcare/FhirIO.java","lineNumber":1221,"sourceCode":"       * @param context the context\n       * @throws IOException the io exception\n       */\n      @ProcessElement\n      public void addToFile(ProcessContext context, BoundedWindow window) throws IOException {\n        this.window = window;\n        String httpBody = context.element();\n        try {\n          // This will error if not valid JSON an convert Pretty JSON to raw JSON.\n          Object data = this.mapper.readValue(httpBody, Object.class);\n          String ndJson = this.mapper.writeValueAsString(data) + \"\\n\";\n          this.ndJsonChannel.write(ByteBuffer.wrap(ndJson.getBytes(StandardCharsets.UTF_8)));\n        } catch (JsonProcessingException e) {\n          String resource =\n              String.format(\n                  \"Failed to parse payload: %s as json at: %s : %s.\"\n                      + \"Dropping resource from batch import.\",\n                  httpBody, e.getLocation().getCharOffset(), e.getMessage());\n          LOG.warn(\"{}\", resource);\n          context.output(\n              Write.FAILED_BODY, HealthcareIOError.of(httpBody, new IOException(resource)));\n        }\n      }\n\n      /**\n       * Close file.\n       *\n       * @param context the context\n       * @throws IOException the io exception\n       */\n      @FinishBundle\n      public void closeFile(FinishBundleContext context) throws IOException {\n        // Write the file with all elements in this batch to GCS.\n        ndJsonChannel.close();\n        context.output(resourceId, window.maxTimestamp(), window);\n      }\n    }","sourceCodeStart":1203,"sourceCodeEnd":1239,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/healthcare/FhirIO.java#L1203-L1239","documentation":"FhirIO's batch import body formatting expects each input element to be valid JSON that can be wrapped into an HTTP body part. When Jackson fails to parse/serialize the payload (JsonProcessingException), the element is logged with the character offset of the parse failure, emitted to Write.FAILED_BODY as a HealthcareIOError, and dropped from the batch import — the pipeline continues without it.","triggerScenarios":"Supplying FhirIO.write().resources() with strings that are not valid JSON documents (truncated JSON, NDJSON lines concatenated, HTML error pages, empty strings, or payloads that are valid but not FHIR-shaped enough for the body formatter).","commonSituations":"Importing from a text source where records were not validated as JSON; encoding issues corrupting payloads; concatenating multiple JSON objects without delimiters; upstream systems emitting partial records.","solutions":["Validate each payload parses as JSON before feeding it to FhirIO (e.g. new ObjectMapper().readTree(payload)).","Route and inspect the FAILED_BODY PCollection to find malformed payloads and their offsets.","Fix upstream serialization so each element is exactly one complete JSON FHIR resource.","If NDJSON, split records into individual JSON strings first."],"exampleFix":"// before\nPCollection<String> raw = pipeline.apply(TextIO.read().from(\"import.ndjson\")); // one big blob line\n// after\nPCollection<String> records = pipeline.apply(TextIO.read().from(\"import.ndjson\")).apply(ParDo.of(new SplitNdJsonFn()));\nrecords.apply(\"Validate\", ParDo.of(new JsonValidationFn())).apply(FhirIO.write().resources());","handlingStrategy":"validation","validationCode":"// Java: validate JSON before feeding FhirIO.write().resources()\nprivate static final com.fasterxml.jackson.databind.ObjectMapper OM = new com.fasterxml.jackson.databind.ObjectMapper();\npublic static boolean isValidJson(String s) {\n  try { OM.readTree(s); return true; } catch (Exception e) { return false; }\n}","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Validate every payload parses as JSON before the write transform.","Ensure each input element is exactly one complete FHIR JSON resource.","Monitor the Write.FAILED_BODY PCollection.","Guard against upstream concatenation/NDJSON blobs."],"tags":["gcp-healthcare","fhir","json-parse","dead-letter"],"backgroundTag":"json-parse-error","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}