apache/iceberg · error · IOException

Unknown version:

Error message

Unknown version: 

What it means

IcebergEnumeratorPositionSerializer.deserialize only supports serialization version 1; encountering a different version byte in checkpoint state throws IOException. The serialized enumerator position was written by an incompatible serializer version.

Source

Thrown at flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/source/enumerator/IcebergEnumeratorPositionSerializer.java:53

      ThreadLocal.withInitial(() -> new DataOutputSerializer(128));

  @Override
  public int getVersion() {
    return VERSION;
  }

  @Override
  public byte[] serialize(IcebergEnumeratorPosition position) throws IOException {
    return serializeV1(position);
  }

  @Override
  public IcebergEnumeratorPosition deserialize(int version, byte[] serialized) throws IOException {
    switch (version) {
      case 1:
        return deserializeV1(serialized);
      default:
        throw new IOException("Unknown version: " + version);
    }
  }

  private byte[] serializeV1(IcebergEnumeratorPosition position) throws IOException {
    DataOutputSerializer out = SERIALIZER_CACHE.get();
    out.writeBoolean(position.snapshotId() != null);
    if (position.snapshotId() != null) {
      out.writeLong(position.snapshotId());
    }
    out.writeBoolean(position.snapshotTimestampMs() != null);
    if (position.snapshotTimestampMs() != null) {
      out.writeLong(position.snapshotTimestampMs());
    }
    byte[] result = out.getCopyOfBuffer();
    out.clear();
    return result;
  }

View on GitHub (pinned to 86d9c8fc54)

Solutions

1. Restore the checkpoint/savepoint with the same or newer connector version that wrote it.
2. Upgrade the connector rather than downgrading, or start a fresh job without state.
3. If downgrade is required, let the job run from a new checkpoint written by the target version first.

Example fix

// before
# downgrade connector to v1.20 older patch and restore new checkpoint
// after
# keep connector at the version that wrote the checkpoint, or restart without state

try {
  position = positionSerializer.deserialize(version, bytes);
} catch (IOException e) {
  LOG.warn("Incompatible enumerator position state, starting fresh", e);
  position = null; // fall back to configured starting strategy
}

When it happens

Trigger: Restoring a Flink checkpoint/savepoint whose enumerator state bytes were written with a version other than 1 (e.g., newer connector restoring under an older connector).

Common situations: Downgrading the Iceberg Flink connector after upgrading; restoring state across incompatible connector releases.

Understand the failure class

Background: "is not a compatible type" / "cannot merge" errors: when a value's type doesn't match what the library requires — this error's family across 65 libraries.

Related errors

• Unknown version:
• Unknown serialize version:
• Unrecognized version or corrupt state: <version>
• Unknown serialize version: ${version}
• Unrecognized version or corrupt state: ${version}

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/fe79ad9293902cf0. Report an issue: GitHub.