apache/beam · warning

Error fetching Fhir resource with ID

Error message

Error fetching Fhir resource with ID {} writing to Dead Letter Queue. 

What it means

FhirIO.Read (fhir_search/fetch by resource ID) fetches each FHIR resource by ID via the Healthcare API. If the fetch throws any exception (HTTP error, missing resource, auth failure, network error), the counter READ_RESOURCE_ERRORS is incremented, this warning is logged, and the failing resource ID is emitted to the DEAD_LETTER output as a HealthcareIOError instead of failing the pipeline.

Solutions

1. Inspect the DEAD_LETTER PCollection (HealthcareIOError) for the failing IDs and the wrapped exception cause.
2. Grant the pipeline service account roles/healthcare.fhirResourceReader on the dataset/store.
3. Validate store paths (projects/{p}/locations/{l}/datasets/{d}/fhirStores/{s}) and resource ID formats before the read.
4. Retry transient failures; re-run the dead-lettered IDs in a follow-up job after fixing the cause.

Example fix

// before (fail silently to DLQ, ignore)
// after
PCollection<HealthcareIOError<String>> dlq = result.get(FhirIO.Read.DEAD_LETTER);
dlq.apply(ParDo.of(new LogAndAlertOnErrorDoFn()));

// Java: validate resource ID shape before Read
if (!resourceId.matches("projects/[^/]+/locations/[^/]+/datasets/[^/]+/fhirStores/[^/]+/fhir/[A-Za-z]+/[^"]+")) {
  throw new IllegalArgumentException("Invalid FHIR resource ID: " + resourceId);
}

// Consume the dead letter output
PCollection<HealthcareIOError<String>> dlq = fetched.get(FhirIO.Read.DEAD_LETTER);
dlq.apply(ParDo.of(new DoFn<HealthcareIOError<String>, Void>() {
  @ProcessElement
  public void process(ProcessContext c) {
    HealthcareIOError<String> err = c.element();
    LOG.error("FHIR fetch failed for {}: {}", err.getDataResource(), err.getErrorMessage());
  }
}));

When it happens

Trigger: Calling FhirIO.read().fhirStore(store) with resource IDs that don't exist, have been deleted, are in a store the caller lacks fhir.resources.get permission on, or when the Healthcare API endpoint is unreachable/transiently failing.

Common situations: Downstream search results feeding stale IDs; pipeline credentials missing roles/healthcare.fhirResourceReader; store URL typos; resources deleted between search and fetch; transient 429/5xx from the API.

Understand the failure class

Background: "API request failed": what wrapped HTTP errors from external APIs mean and how to find the real cause — this error's family across 29 libraries.

Related errors

• Error executing GetPatientEverything: FHIR resources…
• Error fetching HL7v2 message with ID
• Error search FHIR resources writing to Dead Letter Queue.
• Failed to parse payload
• Failed to import with error. Moving to deadletter path

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/7169a77458585660. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/healthcare/FhirIO.java:612

        public void instantiateHealthcareClient() throws IOException {
          this.client = new HttpHealthcareApiClient();
          this.mapper = new ObjectMapper();
        }

        /**
         * Process element.
         *
         * @param context the context
         */
        @ProcessElement
        public void processElement(ProcessContext context) {
          String resourceId = context.element();
          String resource = null;
          try {
            resource = java.util.Objects.requireNonNull(fetchResource(this.client, resourceId));
          } catch (Exception e) {
            READ_RESOURCE_ERRORS.inc();
            LOG.warn(
                "Error fetching Fhir resource with ID {} writing to Dead Letter Queue. ",
                resourceId,
                e);
            context.output(FhirIO.Read.DEAD_LETTER, HealthcareIOError.of(resourceId, e));
          }
          if (resource != null) {
            context.output(resource);
          }
        }

        private String fetchResource(HealthcareApiClient client, String resourceName)
            throws IOException, IllegalArgumentException {
          long startTime = Instant.now().toEpochMilli();

          HttpBody resource = client.readFhirResource(resourceName);
          READ_RESOURCE_LATENCY_MS.update(Instant.now().toEpochMilli() - startTime);

          if (resource == null) {

View on GitHub (pinned to 12126d8942)