apache/flink · error · IOException

Cannot clean commit: Staging file does not exist.

Error message

Cannot clean commit: Staging file does not exist.

What it means

Thrown by the Committer inside AzureBlobFsRecoverableDataOutputStream when calling fs.getFileStatus(src) on the staging (temp) file throws an IOException that is NOT FileNotFoundException (that case is handled and leaves srcStatus null). The wrapper message is misleading: the staging file may exist; an I/O error while stat-ing it caused the failure.

Source

Thrown at flink-filesystems/flink-azure-fs-hadoop/src/main/java/org/apache/flink/fs/azurefs/AzureBlobFsRecoverableDataOutputStream.java:262

        private final HadoopFsRecoverable recoverable;

        ABFSCommitter(FileSystem fs, HadoopFsRecoverable recoverable) {
            this.fs = checkNotNull(fs);
            this.recoverable = checkNotNull(recoverable);
        }

        @Override
        public void commit() throws IOException {
            final Path src = recoverable.tempFile();
            final Path dest = recoverable.targetFile();
            final long expectedLength = recoverable.offset();
            FileStatus srcStatus = null;
            try {
                srcStatus = fs.getFileStatus(src);
            } catch (FileNotFoundException fnfe) {
                // srcStatus will be null
            } catch (IOException e) {
                throw new IOException("Cannot clean commit: Staging file does not exist.");
            }
            if (srcStatus != null) {
                LOG.debug(
                        "The srcStatus is {} and exp length is {}",
                        srcStatus.getLen(),
                        expectedLength);
                if (srcStatus.getLen() != expectedLength) {
                    LOG.error(
                            "The src file {} with length {} does not match the expected length {}",
                            src,
                            srcStatus.getLen(),
                            expectedLength);
                    throw new IOException(
                            "The src file "
                                    + src
                                    + " with length "
                                    + srcStatus.getLen()
                                    + " "

View on GitHub (pinned to 2f3c205e92)

Solutions

Retry the commit (recovery from the last checkpoint re-attempts pending commits)
Refresh or extend storage credentials (account key / SAS lifetime) so they outlive the longest expected checkpoint interval
Raise ABFS client retry/timeout settings in the Hadoop config (fs.azure.* retry policies) to ride out throttling
Do not read the message literally — check the suppressed/cause chain: the original IOException is swallowed, so correlate with the surrounding task log

Defensive patterns

Strategy: retry

Try / catch

try {
    committer.commit();
} catch (java.io.IOException e) {
    // message is misleading: root cause is an I/O error stat-ing the staging file
    // inspect surrounding logs / cause and retry after credential or throttle checks
    scheduleCommitRetry();
}

Prevention

Set storage credential lifetimes longer than the max checkpoint interval
Tune fs.azure retry policies for throttled accounts
Correlate with task logs because the original IOException is swallowed here

When it happens

Trigger: commit()/commitAfterRecovery() on an abfs recoverable sink where the getFileStatus call fails with a network/authentication error: expired SAS token, throttling (ABFS 429/503), or a storage outage — anything that raises IOException besides file-not-found.

Common situations: Long-running jobs whose SAS credential expires before commit; throttled storage accounts during heavy checkpointing; transient Azure availability events.

Related errors

AI-assisted analysis of apache/flink@2f3c205e92 (2026-08-14). Data as JSON: /api/errors/69be41d885e8cba0. Report an issue: GitHub.