apache/flink · error · IllegalArgumentException

can not recover from the pendingFileRecoverable

Error message

can not recover from the pendingFileRecoverable

What it means

Thrown by OutputStreamBasedBucketWriter.recoverForCommit when the provided PendingFileRecoverable is neither OutputStreamBasedPendingFileRecoverable nor OutputStreamBasedInProgressFileRecoverable. The recovery path only understands these two concrete types; any other PendingFileRecoverable implementation cannot be deserialized into a commit-recoverable handle.

Source

Thrown at flink-connectors/flink-file-sink-common/src/main/java/org/apache/flink/streaming/api/functions/sink/filesystem/OutputStreamBasedPartFileWriter.java:168

                    creationTime);
        }

        @Override
        public PendingFile recoverPendingFile(final PendingFileRecoverable pendingFileRecoverable)
                throws IOException {
            final RecoverableWriter.CommitRecoverable commitRecoverable;

            if (pendingFileRecoverable instanceof OutputStreamBasedPendingFileRecoverable) {
                commitRecoverable =
                        ((OutputStreamBasedPendingFileRecoverable) pendingFileRecoverable)
                                .getCommitRecoverable();
            } else if (pendingFileRecoverable
                    instanceof OutputStreamBasedInProgressFileRecoverable) {
                commitRecoverable =
                        ((OutputStreamBasedInProgressFileRecoverable) pendingFileRecoverable)
                                .getResumeRecoverable();
            } else {
                throw new IllegalArgumentException(
                        "can not recover from the pendingFileRecoverable");
            }
            return new OutputStreamBasedPendingFile(
                    recoverableWriter.recoverForCommit(commitRecoverable));
        }

        @Override
        public boolean cleanupInProgressFileRecoverable(
                InProgressFileRecoverable inProgressFileRecoverable) throws IOException {
            final RecoverableWriter.ResumeRecoverable resumeRecoverable =
                    ((OutputStreamBasedInProgressFileRecoverable) inProgressFileRecoverable)
                            .getResumeRecoverable();
            return recoverableWriter.cleanupRecoverableState(resumeRecoverable);
        }

        @Override
        public WriterProperties getProperties() {
            return new WriterProperties(

View on GitHub (pinned to 2f3c205e92)

Solutions

1. Ensure the RecoverableWriter and BucketWriter used at recovery time match those used when the state was originally checkpointed.
2. Re-take the savepoint after changing the underlying filesystem or recoverable writer implementation.
3. Do not mix custom PendingFileRecoverable implementations with the built-in OutputStreamBased recoverables.

public static boolean isRecoverableSupported(PendingFileRecoverable r) {
    return r instanceof OutputStreamBasedPendingFileRecoverable
        || r instanceof OutputStreamBasedInProgressFileRecoverable;
}

if (!isRecoverableSupported(pendingFileRecoverable)) {
    throw new IllegalStateException(
        "Unsupported PendingFileRecoverable type: " + pendingFileRecoverable.getClass());
}
bucketWriter.recoverForCommit(pendingFileRecoverable);

When it happens

Trigger: Calling BucketWriter.recoverForCommit() with a PendingFileRecoverable produced by a different BucketWriter implementation or a corrupt/incompatible serialized state.

Common situations: Restoring a savepoint taken with a different RecoverableWriter (e.g. switching filesystem types from HDFS to S3 without re-taking the savepoint); a custom RecoverableWriter whose recoverable types are incompatible with OutputStreamBasedPartFileWriter.

Related errors

• Bulk Part Writers do not support "pause and resume" operatio
• Writer has already been opened as {writeType} type, but tryi
• Unrecognized version or corrupt state: {version}
• Unable to recover the job as the expected {} file is not fou
• Unable to create recoverable outputstream as length of file

AI-assisted analysis of apache/flink@2f3c205e92 (2026-08-14). Data as JSON: /api/errors/2587d93868461fb3. Report an issue: GitHub.