{"record":{"id":"2c3a48bbcec33ef5","repo":"apache/flink","slug":"committing-file-by-rename-failed-to","errorCode":null,"errorMessage":"Committing file by rename failed: {} to {}","messagePattern":"Committing file by rename failed: (.+?) to (.+?)","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"flink-filesystems/flink-azure-fs-hadoop/src/main/java/org/apache/flink/fs/azurefs/AzureBlobFsRecoverableDataOutputStream.java","lineNumber":287,"sourceCode":"                if (srcStatus.getLen() != expectedLength) {\n                    LOG.error(\n                            \"The src file {} with length {} does not match the expected length {}\",\n                            src,\n                            srcStatus.getLen(),\n                            expectedLength);\n                    throw new IOException(\n                            \"The src file \"\n                                    + src\n                                    + \" with length \"\n                                    + srcStatus.getLen()\n                                    + \" \"\n                                    + \"does not match the expected length \"\n                                    + expectedLength);\n                }\n                try {\n                    fs.rename(src, dest);\n                } catch (IOException e) {\n                    throw new IOException(\n                            \"Committing file by rename failed: \" + src + \" to \" + dest, e);\n                }\n            } else if (!fs.exists(dest)) {\n                // neither exists - that can be a sign of\n                //   - (1) a serious problem (file system loss of data)\n                //   - (2) a recovery of a savepoint that is some time old and the users\n                //         removed the files in the meantime.\n                throw new IOException(\n                        \"Unrecoverable exception while trying to recover \"\n                                + recoverable.tempFile());\n            }\n        }\n\n        @Override\n        public void commitAfterRecovery() throws IOException {\n            commit();\n        }\n","sourceCodeStart":269,"sourceCodeEnd":305,"githubUrl":"https://github.com/apache/flink/blob/2f3c205e9266cb30240eb7f4fdab15cad629a70f/flink-filesystems/flink-azure-fs-hadoop/src/main/java/org/apache/flink/fs/azurefs/AzureBlobFsRecoverableDataOutputStream.java#L269-L305","documentation":"Thrown by the AzureBlobFsRecoverableDataOutputStream Committer when fs.rename(src, dest) throws an IOException while promoting the staging file to its final path during commit(). The original exception is chained as the cause, so the real reason (permissions, lease, throttling) is on the cause, not the message.","triggerScenarios":"commit()/commitAfterRecovery() where the final rename in ABFS fails: destination blob under a lease, credential lacking delete permission (ABFS rename = copy + delete), path already exists as a directory, or transient Azure errors.","commonSituations":"SAS token without delete rights; a reader holding a lease on the target blob; storage throttling during mass commit of many checkpoint files.","solutions":["Inspect the cause exception in the stack trace to classify permission vs lease vs throttling","Retry the commit from the same checkpoint — commits are idempotent (rename overwrites)","Grant write+delete permissions and use full account keys or role-based access instead of narrow SAS tokens","Enable/raise fs.azure retry configuration for throttling scenarios"],"exampleFix":null,"handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"try {\n    committer.commit();\n} catch (java.io.IOException e) {\n    // cause chain holds the real rename failure (lease/permission/throttle)\n    log.error(\"Commit rename failed\", e.getCause());\n    retryCommitWithBackoff(); // rename-overwrite is idempotent\n}","preventionTips":["Use full-permission credentials (write+delete) for ABFS sinks","Release leases on target paths before recovery","Batch/pace commits to avoid ABFS throttling storms"],"tags":["azure","abfs","commit","rename","permissions"],"backgroundTag":null,"analyzedSha":"2f3c205e9266cb30240eb7f4fdab15cad629a70f","analyzedAt":"2026-08-14T08:48:24.518Z","schemaVersion":2},"datasetVersion":"2026-08-14T10:17:34.591Z"}