apache/flink · error · IOException

Unrecognized version or corrupt state: {version}

Error message

Unrecognized version or corrupt state: {version}

What it means

Thrown during deserialization of a FileSinkCommittable when the version number is not 1 or 2. The serializer supports two versions: v1 (legacy, pre-bucket-id) and v2 (current, with bucket id and compacted file cleanup). An unrecognized version indicates state incompatibility or corruption.

Source

Thrown at flink-connectors/flink-connector-files/src/main/java/org/apache/flink/connector/file/sink/FileSinkCommittableSerializer.java:82

        DataOutputSerializer out = new DataOutputSerializer(256);
        out.writeInt(MAGIC_NUMBER);
        serializeV2(committable, out);
        return out.getCopyOfBuffer();
    }

    @Override
    public FileSinkCommittable deserialize(int version, byte[] serialized) throws IOException {
        DataInputDeserializer in = new DataInputDeserializer(serialized);

        switch (version) {
            case 1:
                validateMagicNumber(in);
                return deserializeV1(in);
            case 2:
                validateMagicNumber(in);
                return deserializeV2(in);
            default:
                throw new IOException("Unrecognized version or corrupt state: " + version);
        }
    }

    private void serializeV2(FileSinkCommittable committable, DataOutputView dataOutputView)
            throws IOException {
        dataOutputView.writeUTF(committable.getBucketId());
        if (committable.hasPendingFile()) {
            dataOutputView.writeBoolean(true);
            SimpleVersionedSerialization.writeVersionAndSerialize(
                    pendingFileSerializer, committable.getPendingFile(), dataOutputView);
        } else {
            dataOutputView.writeBoolean(false);
        }

        if (committable.hasInProgressFileToCleanup()) {
            dataOutputView.writeBoolean(true);
            SimpleVersionedSerialization.writeVersionAndSerialize(
                    inProgressFileSerializer,

View on GitHub (pinned to 2f3c205e92)

Solutions

1. Ensure the Flink version creating the checkpoint is compatible with the version restoring it.
2. If upgrading across major versions, follow the Flink upgrade and state compatibility guide.
3. If the state is genuinely corrupt or incompatible, start from a clean state without restoring from the savepoint.
4. Check the version number in the error message to determine the mismatch direction.

// Check serializer version compatibility before restore
FileSinkCommittableSerializer serializer = ...;
int currentVersion = serializer.getVersion();
// Ensure the checkpoint was written with version <= currentVersion
// Version 1 = legacy format, Version 2 = current with bucketId + compacted cleanup

try {
    serializer.deserialize(version, data);
} catch (IOException e) {
    if (e.getMessage().startsWith("Unrecognized version or corrupt state")) {
        // version mismatch — cannot restore from this checkpoint
        // start fresh or use a compatible savepoint
    }
    throw e;
}

When it happens

Trigger: Restoring a checkpoint or savepoint that was written by a different version of Flink with an incompatible committable serialization format; corrupted state data where the version byte was overwritten; manually tampered-with state.

Common situations: Upgrading or downgrading Flink across a version boundary that changed the committable serializer version; restoring from a savepoint produced by a different sink implementation; state backend corruption.

Related errors

• Corrupt data: Unexpected magic number %08X
• Unrecognized version or corrupt state: {version}
• Corrupt data: Unexpected magic number %08X
• Unrecognized version or corrupt state: {version}
• Unrecognized version: {}

AI-assisted analysis of apache/flink@2f3c205e92 (2026-08-14). Data as JSON: /api/errors/efa705ca06748d3c. Report an issue: GitHub.