apache/flink · error · IOException

Serialized data with version %d cannot be read by serializer

Error message

Serialized data with version %d cannot be read by serializer with version %d

What it means

Thrown by GSCommitRecoverableSerializer.deserialize when the persisted recoverable-commit data was written by a NEWER serializer (version number higher than SERIALIZER_VERSION of the code reading it). The GCS recoverable writer stores a version tag with each serialized GSCommitRecoverable so that older code can refuse data it cannot interpret instead of corrupting it. This is a forward-incompatibility guard for checkpoint/savepoint metadata of in-flight file sinks.

Source

Thrown at flink-filesystems/flink-gs-fs-hadoop/src/main/java/org/apache/flink/fs/gs/writer/GSCommitRecoverableSerializer.java:141

            long lsbValue = dataInputStream.readLong();
            UUID componentObjectId = new UUID(msbValue, lsbValue);
            componentObjectIds.add(componentObjectId);
        }

        GSCommitRecoverable recoverable =
                new GSCommitRecoverable(finalBlobIdentifier, componentObjectIds);
        LOGGER.trace("Deserialized commit recoverable {}", recoverable);
        return recoverable;
    }

    @Override
    public GSCommitRecoverable deserialize(int version, byte[] serialized) throws IOException {
        Preconditions.checkArgument(version > 0);
        Preconditions.checkNotNull(serialized);

        // ensure this serializer can deserialize data with this version
        if (version > SERIALIZER_VERSION) {
            throw new IOException(
                    String.format(
                            "Serialized data with version %d cannot be read by serializer with version %d",
                            version, SERIALIZER_VERSION));
        }

        try (ByteArrayInputStream inputStream = new ByteArrayInputStream(serialized)) {
            try (DataInputStream dataInputStream = new DataInputStream(inputStream)) {
                return deserializeCommitRecoverable(dataInputStream);
            }
        }
    }
}

View on GitHub (pinned to 2f3c205e92)

Solutions

Roll forward: run the job with the same or newer flink-gs-fs-hadoop / Flink version that wrote the checkpoint
Verify the plugin jar version in lib/ matches the version that produced the checkpoint (no mixed jars on the classpath)
If the state is disposable, discard the stale pending-commit state and restart the job from a fresh savepoint or clear the in-flight data (accepting data loss for those pending files)

Example fix

// before: restore attempted with older plugin
// after: use the same/newer plugin version that serialized the state
// java -lib flink-gs-fs-hadoop-<newer-version>.jar matching the checkpoint

Defensive patterns

Strategy: try-catch

Validate before calling

// before restore, check the persisted version if accessible
if (storedVersion > GSCommitRecoverableSerializer.SERIALIZER_VERSION) {
    throw new IllegalStateException("State written by newer GCS serializer; upgrade plugin first");
}

Try / catch

try {
    committer = serializer.deserialize(version, bytes);
} catch (IOException e) {
    if (e.getMessage().contains("cannot be read by serializer")) {
        // version mismatch: require plugin roll-forward, do not retry
        throw new IllegalStateException("GCS recoverable state version too new", e);
    }
    throw e;
}

Prevention

Pin one flink-gs-fs-hadoop version across all environments
Before downgrading Flink/plugin, finish or clear all pending file-sink commits
Treat serializer-version errors as configuration failures, not transient I/O

When it happens

Trigger: Restoring a checkpoint/savepoint (or resuming a pending commit) whose GSCommitRecoverable was serialized by a newer flink-gs-fs-hadoop plugin, i.e. version > SERIALIZER_VERSION of the jar currently on the classpath; then calling GSCommitRecoverableSerializer.deserialize(version, bytes).

Common situations: Downgrading the Flink GCS filesystem plugin or Flink version after a job already checkpointed pending commit recoverables; mixing plugin versions between the job that wrote the state and the job that reads it; reading state produced by a newer Flink in a rollback during an upgrade.

Related errors

AI-assisted analysis of apache/flink@2f3c205e92 (2026-08-14). Data as JSON: /api/errors/c4f889ce19ff8c10. Report an issue: GitHub.