{"record":{"id":"19e439e570f77256","repo":"apache/beam","slug":"failed-to-import-with-error-moving-to-deadletter-path","errorCode":null,"errorMessage":"Failed to import {} with error. Moving to deadletter path {}","messagePattern":"Failed to import (.+?) with error\\. Moving to deadletter path (.+?)","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/healthcare/FhirIO.java","lineNumber":1354,"sourceCode":"          assert contentStructure != null;\n          Operation operation =\n              client.importFhirResource(\n                  fhirStore.get(), importUri.toString(), contentStructure.name());\n          operation = client.pollOperation(operation, 15000L);\n          incrementLroCounters(\n              operation,\n              IMPORT_OPERATION_SUCCESS,\n              IMPORT_OPERATION_ERRORS,\n              RESOURCES_IMPORTED_SUCCESS,\n              RESOURCES_IMPORTED_ERRORS);\n\n          // Clean up temp files on GCS as they were successfully imported to FHIR store and no\n          // longer needed.\n          FileSystems.delete(tempDestinations);\n        } catch (IOException | InterruptedException e) {\n          ResourceId deadLetterResourceId =\n              FileSystems.matchNewResource(deadLetterGcsPath.get(), true);\n          LOG.warn(\n              \"Failed to import {} with error. Moving to deadletter path {}\",\n              importUri,\n              deadLetterResourceId.toString(),\n              e);\n          IMPORT_OPERATION_ERRORS.inc();\n\n          FileSystems.rename(tempDestinations, deadLetterDestinations);\n          context.output(\n              HealthcareIOError.of(importUri.toString(), e), window.maxTimestamp(), window);\n        }\n      }\n    }\n\n    /** The enum Content structure. */\n    public enum ContentStructure {\n      /** If the content structure is not specified, the default value BUNDLE will be used. */\n      CONTENT_STRUCTURE_UNSPECIFIED,\n      /**","sourceCodeStart":1336,"sourceCodeEnd":1372,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/healthcare/FhirIO.java#L1336-L1372","documentation":"After successfully executing a FHIR store import, cleaning up the temporary GCS staging files failed (IOException or InterruptedException from FileSystems.delete). The pipeline logs the failure and moves the record to the configured dead-letter GCS path rather than failing the pipeline; the IMPORT_OPERATION_ERRORS counter is incremented.","triggerScenarios":"FhirIO.importToFhirStore succeeds but the follow-up FileSystems.delete(tempDestinations) throws IOException (GCS permission issue, transient storage error) or InterruptedException. Requires a deadLetterGcsPath to be configured.","commonSituations":"Service account lacking storage.objects.delete permission on the temp bucket, transient GCS 5xx errors, temp files already deleted by a lifecycle rule, or worker thread interruption during pipeline shutdown.","solutions":["Grant the pipeline service account object deleter (or admin) permission on the temp GCS bucket","Inspect the dead-letter path for the failed import URIs and reprocess them manually","Check GCS bucket lifecycle rules — files may already be gone; deletion errors on missing objects can be treated as success","If only cleanup failed (import succeeded), verify the FHIR store state and safely skip/ignore the dead-lettered record"],"exampleFix":"// before\ntry { FileSystems.delete(tempDestinations); } catch (IOException | InterruptedException e) { ... move to deadletter ... }\n// after\ntry {\n  FileSystems.delete(tempDestinations);\n} catch (IOException e) {\n  LOG.warn(\"Cleanup of temp files failed after successful import of {}; ignoring\", importUri, e);\n}","handlingStrategy":"fallback","validationCode":"// check cleanup permission up front\nboolean canDelete = testDeletePermission(deadLetterBucket, stagingPrefix);\nif (!canDelete) LOG.warn(\"Service account cannot delete temp files; cleanup will fail\");","typeGuard":null,"tryCatchPattern":"try {\n  FileSystems.delete(tempDestinations);\n} catch (IOException e) {\n  LOG.warn(\"Import succeeded but temp cleanup failed for {}; import result is kept\", importUri, e);\n  // do not dead-letter a successful import\n}","preventionTips":["Grant storage.objects.delete (or legacy bucketWriter) on the temp bucket","Set a GCS lifecycle rule so orphaned temp files expire even if cleanup fails","Always configure deadLetterGcsPath so failed imports are preserved for replay","Distinguish import failure from cleanup failure before dead-lettering"],"tags":["gcp","healthcare","fhir","gcs","dead-letter"],"backgroundTag":"file-delete-failed","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}