{"record":{"id":"69be41d885e8cba0","repo":"apache/flink","slug":"cannot-clean-commit-staging-file-does-not-exist","errorCode":null,"errorMessage":"Cannot clean commit: Staging file does not exist.","messagePattern":"Cannot clean commit: Staging file does not exist\\.","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"flink-filesystems/flink-azure-fs-hadoop/src/main/java/org/apache/flink/fs/azurefs/AzureBlobFsRecoverableDataOutputStream.java","lineNumber":262,"sourceCode":"        private final HadoopFsRecoverable recoverable;\n\n        ABFSCommitter(FileSystem fs, HadoopFsRecoverable recoverable) {\n            this.fs = checkNotNull(fs);\n            this.recoverable = checkNotNull(recoverable);\n        }\n\n        @Override\n        public void commit() throws IOException {\n            final Path src = recoverable.tempFile();\n            final Path dest = recoverable.targetFile();\n            final long expectedLength = recoverable.offset();\n            FileStatus srcStatus = null;\n            try {\n                srcStatus = fs.getFileStatus(src);\n            } catch (FileNotFoundException fnfe) {\n                // srcStatus will be null\n            } catch (IOException e) {\n                throw new IOException(\"Cannot clean commit: Staging file does not exist.\");\n            }\n            if (srcStatus != null) {\n                LOG.debug(\n                        \"The srcStatus is {} and exp length is {}\",\n                        srcStatus.getLen(),\n                        expectedLength);\n                if (srcStatus.getLen() != expectedLength) {\n                    LOG.error(\n                            \"The src file {} with length {} does not match the expected length {}\",\n                            src,\n                            srcStatus.getLen(),\n                            expectedLength);\n                    throw new IOException(\n                            \"The src file \"\n                                    + src\n                                    + \" with length \"\n                                    + srcStatus.getLen()\n                                    + \" \"","sourceCodeStart":244,"sourceCodeEnd":280,"githubUrl":"https://github.com/apache/flink/blob/2f3c205e9266cb30240eb7f4fdab15cad629a70f/flink-filesystems/flink-azure-fs-hadoop/src/main/java/org/apache/flink/fs/azurefs/AzureBlobFsRecoverableDataOutputStream.java#L244-L280","documentation":"Thrown by the Committer inside AzureBlobFsRecoverableDataOutputStream when calling fs.getFileStatus(src) on the staging (temp) file throws an IOException that is NOT FileNotFoundException (that case is handled and leaves srcStatus null). The wrapper message is misleading: the staging file may exist; an I/O error while stat-ing it caused the failure.","triggerScenarios":"commit()/commitAfterRecovery() on an abfs recoverable sink where the getFileStatus call fails with a network/authentication error: expired SAS token, throttling (ABFS 429/503), or a storage outage — anything that raises IOException besides file-not-found.","commonSituations":"Long-running jobs whose SAS credential expires before commit; throttled storage accounts during heavy checkpointing; transient Azure availability events.","solutions":["Retry the commit (recovery from the last checkpoint re-attempts pending commits)","Refresh or extend storage credentials (account key / SAS lifetime) so they outlive the longest expected checkpoint interval","Raise ABFS client retry/timeout settings in the Hadoop config (fs.azure.* retry policies) to ride out throttling","Do not read the message literally — check the suppressed/cause chain: the original IOException is swallowed, so correlate with the surrounding task log"],"exampleFix":null,"handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"try {\n    committer.commit();\n} catch (java.io.IOException e) {\n    // message is misleading: root cause is an I/O error stat-ing the staging file\n    // inspect surrounding logs / cause and retry after credential or throttle checks\n    scheduleCommitRetry();\n}","preventionTips":["Set storage credential lifetimes longer than the max checkpoint interval","Tune fs.azure retry policies for throttled accounts","Correlate with task logs because the original IOException is swallowed here"],"tags":["azure","abfs","commit","io","misleading-message"],"backgroundTag":null,"analyzedSha":"2f3c205e9266cb30240eb7f4fdab15cad629a70f","analyzedAt":"2026-08-14T08:48:24.518Z","schemaVersion":2},"datasetVersion":"2026-08-14T10:17:34.591Z"}