apache/beam · warning

Failed to import with error. Moving to deadletter path

Error message

Failed to import {} with error. Moving to deadletter path {}

What it means

After successfully executing a FHIR store import, cleaning up the temporary GCS staging files failed (IOException or InterruptedException from FileSystems.delete). The pipeline logs the failure and moves the record to the configured dead-letter GCS path rather than failing the pipeline; the IMPORT_OPERATION_ERRORS counter is incremented.

Solutions

1. Grant the pipeline service account object deleter (or admin) permission on the temp GCS bucket
2. Inspect the dead-letter path for the failed import URIs and reprocess them manually
3. Check GCS bucket lifecycle rules — files may already be gone; deletion errors on missing objects can be treated as success
4. If only cleanup failed (import succeeded), verify the FHIR store state and safely skip/ignore the dead-lettered record

Example fix

// before
try { FileSystems.delete(tempDestinations); } catch (IOException | InterruptedException e) { ... move to deadletter ... }
// after
try {
  FileSystems.delete(tempDestinations);
} catch (IOException e) {
  LOG.warn("Cleanup of temp files failed after successful import of {}; ignoring", importUri, e);
}

// check cleanup permission up front
boolean canDelete = testDeletePermission(deadLetterBucket, stagingPrefix);
if (!canDelete) LOG.warn("Service account cannot delete temp files; cleanup will fail");

try {
  FileSystems.delete(tempDestinations);
} catch (IOException e) {
  LOG.warn("Import succeeded but temp cleanup failed for {}; import result is kept", importUri, e);
  // do not dead-letter a successful import
}

When it happens

Trigger: FhirIO.importToFhirStore succeeds but the follow-up FileSystems.delete(tempDestinations) throws IOException (GCS permission issue, transient storage error) or InterruptedException. Requires a deadLetterGcsPath to be configured.

Common situations: Service account lacking storage.objects.delete permission on the temp bucket, transient GCS 5xx errors, temp files already deleted by a lifecycle rule, or worker thread interruption during pipeline shutdown.

Related errors

• Failed to ingest message
• Could not find file
• could not resolve to a temp directory for import batch files
• DicomSearch failed with status
• Error constructing default value for gcpTempLocation…

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/19e439e570f77256. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/healthcare/FhirIO.java:1354

          assert contentStructure != null;
          Operation operation =
              client.importFhirResource(
                  fhirStore.get(), importUri.toString(), contentStructure.name());
          operation = client.pollOperation(operation, 15000L);
          incrementLroCounters(
              operation,
              IMPORT_OPERATION_SUCCESS,
              IMPORT_OPERATION_ERRORS,
              RESOURCES_IMPORTED_SUCCESS,
              RESOURCES_IMPORTED_ERRORS);

          // Clean up temp files on GCS as they were successfully imported to FHIR store and no
          // longer needed.
          FileSystems.delete(tempDestinations);
        } catch (IOException | InterruptedException e) {
          ResourceId deadLetterResourceId =
              FileSystems.matchNewResource(deadLetterGcsPath.get(), true);
          LOG.warn(
              "Failed to import {} with error. Moving to deadletter path {}",
              importUri,
              deadLetterResourceId.toString(),
              e);
          IMPORT_OPERATION_ERRORS.inc();

          FileSystems.rename(tempDestinations, deadLetterDestinations);
          context.output(
              HealthcareIOError.of(importUri.toString(), e), window.maxTimestamp(), window);
        }
      }
    }

    /** The enum Content structure. */
    public enum ContentStructure {
      /** If the content structure is not specified, the default value BUNDLE will be used. */
      CONTENT_STRUCTURE_UNSPECIFIED,
      /**

View on GitHub (pinned to 12126d8942)