{"record":{"id":"ec51cdfe8608ff06","repo":"apache/flink","slug":"unrecoverable-exception-while-trying-to-recover","errorCode":null,"errorMessage":"Unrecoverable exception while trying to recover {}","messagePattern":"Unrecoverable exception while trying to recover (.+?)","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"critical","filePath":"flink-filesystems/flink-azure-fs-hadoop/src/main/java/org/apache/flink/fs/azurefs/AzureBlobFsRecoverableDataOutputStream.java","lineNumber":295,"sourceCode":"                                    + src\n                                    + \" with length \"\n                                    + srcStatus.getLen()\n                                    + \" \"\n                                    + \"does not match the expected length \"\n                                    + expectedLength);\n                }\n                try {\n                    fs.rename(src, dest);\n                } catch (IOException e) {\n                    throw new IOException(\n                            \"Committing file by rename failed: \" + src + \" to \" + dest, e);\n                }\n            } else if (!fs.exists(dest)) {\n                // neither exists - that can be a sign of\n                //   - (1) a serious problem (file system loss of data)\n                //   - (2) a recovery of a savepoint that is some time old and the users\n                //         removed the files in the meantime.\n                throw new IOException(\n                        \"Unrecoverable exception while trying to recover \"\n                                + recoverable.tempFile());\n            }\n        }\n\n        @Override\n        public void commitAfterRecovery() throws IOException {\n            commit();\n        }\n\n        @Override\n        public RecoverableWriter.CommitRecoverable getRecoverable() {\n            return recoverable;\n        }\n    }\n}\n","sourceCodeStart":277,"sourceCodeEnd":312,"githubUrl":"https://github.com/apache/flink/blob/2f3c205e9266cb30240eb7f4fdab15cad629a70f/flink-filesystems/flink-azure-fs-hadoop/src/main/java/org/apache/flink/fs/azurefs/AzureBlobFsRecoverableDataOutputStream.java#L277-L312","documentation":"Thrown by the AzureBlobFsRecoverableDataOutputStream Committer when, at commit time, neither the staging (temp) file nor the final target file exists. The code comments flag exactly two possible meanings: (1) real file-system data loss, or (2) recovering from an old savepoint whose output files were removed afterwards. Commit is impossible either way.","triggerScenarios":"commitAfterRecovery() on a savepoint taken long ago, after the bucket contents were cleaned; or genuine ABFS data loss where both the temp file and an already-committed target vanished.","commonSituations":"Resuming an old savepoint for a file sink whose part files were archived/deleted by policy or manually; storage account corrupted or misconfigured after the checkpoint was taken.","solutions":["If files were intentionally deleted, accept the loss and restart with a fresh sink state (new checkpoint) rather than the stale savepoint","Verify the target path/container still holds the expected part files before resuming an old savepoint","Protect output directories from lifecycle policies and manual cleanup","For suspected data loss, engage Azure support with the timestamps from the task log"],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":"// before resuming an old savepoint, verify referenced files still exist\nif (!fs.exists(recoverable.tempFile()) && !fs.exists(recoverable.targetFile())) {\n    // files already gone: plan a fresh run instead of a doomed recovery\n}","typeGuard":null,"tryCatchPattern":"try {\n    committer.commitAfterRecovery();\n} catch (java.io.IOException e) {\n    if (e.getMessage() != null && e.getMessage().contains(\"Unrecoverable\")) {\n        restartFreshWithoutStaleSavepoint();\n    } else { throw e; }\n}","preventionTips":["Archive or preserve part files for the lifetime of any savepoint you may resume","Document retention windows for output buckets","Dry-run savepoint resumes in a test container against the real bucket"],"tags":["azure","abfs","commit","savepoint-recovery","data-loss"],"backgroundTag":null,"analyzedSha":"2f3c205e9266cb30240eb7f4fdab15cad629a70f","analyzedAt":"2026-08-14T08:48:24.518Z","schemaVersion":2},"datasetVersion":"2026-08-14T10:17:34.591Z"}