{"record":{"id":"6d0ba17ae433a0f0","repo":"apache/flink","slug":"unable-to-recover-rename-operation-failed","errorCode":null,"errorMessage":"Unable to recover. Rename operation failed","messagePattern":"Unable to recover\\. Rename operation failed","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"flink-filesystems/flink-azure-fs-hadoop/src/main/java/org/apache/flink/fs/azurefs/AzureBlobFsRecoverableDataOutputStream.java","lineNumber":209,"sourceCode":"        } catch (IOException e) {\n            LOG.error(\"Unable to recover. Error while deleting the temp file {}\", tempFile);\n            // unable to recover.\n            throw e;\n        }\n        rename(fs, renameTempPath);\n    }\n\n    private void rename(FileSystem fs, Path renameTempPath) throws IOException {\n        LOG.info(\"Renaming the temp rename file {} back to tempFile {}\", renameTempPath, tempFile);\n        try {\n            // Rename by default will overwrite if dest is already found.\n            boolean result = fs.rename(renameTempPath, tempFile);\n            if (!result) {\n                LOG.error(\n                        \"Unable to recover. Rename operation failed {} to {}\",\n                        renameTempPath,\n                        tempFile);\n                throw new IOException(\"Unable to recover. Rename operation failed\");\n            } else {\n                LOG.info(\"Rename was successful\");\n            }\n        } catch (IOException e) {\n            LOG.error(\n                    \"Unable to recover. Renaming of tempFile did not happen after truncating {} to {}\",\n                    renameTempPath,\n                    tempFile);\n            throw e;\n        }\n    }\n\n    @Override\n    public void sync() throws IOException {\n        out.hsync();\n    }\n\n    @Override","sourceCodeStart":191,"sourceCodeEnd":227,"githubUrl":"https://github.com/apache/flink/blob/2f3c205e9266cb30240eb7f4fdab15cad629a70f/flink-filesystems/flink-azure-fs-hadoop/src/main/java/org/apache/flink/fs/azurefs/AzureBlobFsRecoverableDataOutputStream.java#L191-L227","documentation":"Thrown by AzureBlobFsRecoverableDataOutputStream.rename() during recovery when Hadoop's FileSystem.rename(renameTempPath, tempFile) returns false (ABFS reports failure without throwing). The '.rename' sidecar could not be moved back over the temp file, so the recoverable stream cannot be reconstructed.","triggerScenarios":"Recovery path where the '.rename' file exists with matching length but the rename call fails: insufficient permissions (SAS token without write/delete), a lease on the destination blob, or transient ABFS errors.","commonSituations":"Sink configured with a restricted SAS credential or read-only storage key; concurrent recovery attempts from two JobManagers fighting over the same files; transient Azure control-plane failures.","solutions":["Retry job recovery — transient ABFS rename failures usually clear","Grant the credential used by the sink write AND delete rights on the container (rename in ABFS needs delete permission)","Ensure only one active job attempt recovers the same checkpoint (no zombie JobManager)","Check for blob leases on tempFile and release them"],"exampleFix":null,"handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"try {\n    writer.recover(recoverable).close();\n} catch (java.io.IOException e) {\n    if (e.getMessage() != null && e.getMessage().contains(\"Rename operation failed\")) {\n        retryWithBackoff(); // transient ABFS rename failures often clear\n    } else { throw e; }\n}","preventionTips":["Give sink credentials write AND delete permissions on the container","Avoid concurrent recovery attempts over the same files","Clear blob leases on temp paths before planned recoveries"],"tags":["azure","abfs","rename","recovery","permissions"],"backgroundTag":null,"analyzedSha":"2f3c205e9266cb30240eb7f4fdab15cad629a70f","analyzedAt":"2026-08-14T08:48:24.518Z","schemaVersion":2},"datasetVersion":"2026-08-14T10:17:34.591Z"}