apache/flink · error · IOException

Unrecognized version or corrupt state: {version}

Error message

Unrecognized version or corrupt state: {version}

What it means

Thrown by OutputStreamBasedInProgressFileRecoverableSerializer.deserialize when the serialized state version is not 1 or 2. The serializer knows how to read exactly these two versions; any other version indicates either a newer format produced by a different Flink version or genuinely corrupt state bytes.

Source

Thrown at flink-connectors/flink-file-sink-common/src/main/java/org/apache/flink/streaming/api/functions/sink/filesystem/OutputStreamBasedPartFileWriter.java:397

            dataOutputSerializer.writeInt(MAGIC_NUMBER);
            serializeV2(outputStreamBasedInProgressRecoverable, dataOutputSerializer);
            return dataOutputSerializer.getCopyOfBuffer();
        }

        @Override
        public InProgressFileRecoverable deserialize(int version, byte[] serialized)
                throws IOException {
            switch (version) {
                case 1:
                    DataInputView dataInputView = new DataInputDeserializer(serialized);
                    validateMagicNumber(dataInputView);
                    return deserializeV1(dataInputView);
                case 2:
                    dataInputView = new DataInputDeserializer(serialized);
                    validateMagicNumber(dataInputView);
                    return deserializeV2(dataInputView);
                default:
                    throw new IOException("Unrecognized version or corrupt state: " + version);
            }
        }

        public SimpleVersionedSerializer<RecoverableWriter.ResumeRecoverable>
                getResumeSerializer() {
            return resumeSerializer;
        }

        private void serializeV2(
                final OutputStreamBasedInProgressFileRecoverable
                        outputStreamBasedInProgressRecoverable,
                final DataOutputView dataOutputView)
                throws IOException {
            boolean pathAvailable = outputStreamBasedInProgressRecoverable.targetPath != null;
            dataOutputView.writeBoolean(pathAvailable);
            if (pathAvailable) {
                dataOutputView.writeUTF(
                        outputStreamBasedInProgressRecoverable.targetPath.toUri().toString());

View on GitHub (pinned to 2f3c205e92)

Solutions

  1. Do not downgrade Flink below the version that wrote the checkpoint; take a fresh savepoint if you need to downgrade.
  2. If the error is from corruption, re-run the pipeline from the last known-good checkpoint without restoring the damaged one.
  3. Verify that the checkpoint was produced by the same File Sink implementation (StreamingFileSink vs FileSink) being used for restore.
Defensive patterns

Strategy: try-catch

Try / catch

try {
    serializer.deserialize(version, bytes);
} catch (IOException e) {
    if (e.getMessage().startsWith("Unrecognized version or corrupt state")) {
        LOG.error("Checkpoint state version {} not supported by this serializer version", version);
        // cannot recover — restart from a compatible checkpoint
    }
    throw e;
}

Prevention

When it happens

Trigger: Restoring a checkpoint or savepoint where the in-progress file recoverable state was written with a serializer version that the current code does not recognize (not 1 or 2).

Common situations: Downgrading Flink to an older version that does not support version 2 of the serializer; or restoring a checkpoint from a different file sink implementation.

Related errors

AI-assisted analysis of apache/flink@2f3c205e92 (2026-08-14). Data as JSON: /api/errors/5c255e459e28b7fa. Report an issue: GitHub.