apache/iceberg · error · IOException

Unknown version: {version}

Error message

Unknown version: {version}

What it means

IcebergEnumeratorPositionSerializer.deserialize throws IOException when the serialized enumerator position blob carries a version byte the deserializer doesn't know (only version 1 exists). This guards state compatibility: a position written by an incompatible writer cannot be safely read.

Source

Thrown at flink/v2.1/flink/src/main/java/org/apache/iceberg/flink/source/enumerator/IcebergEnumeratorPositionSerializer.java:53

      ThreadLocal.withInitial(() -> new DataOutputSerializer(128));

  @Override
  public int getVersion() {
    return VERSION;
  }

  @Override
  public byte[] serialize(IcebergEnumeratorPosition position) throws IOException {
    return serializeV1(position);
  }

  @Override
  public IcebergEnumeratorPosition deserialize(int version, byte[] serialized) throws IOException {
    switch (version) {
      case 1:
        return deserializeV1(serialized);
      default:
        throw new IOException("Unknown version: " + version);
    }
  }

  private byte[] serializeV1(IcebergEnumeratorPosition position) throws IOException {
    DataOutputSerializer out = SERIALIZER_CACHE.get();
    out.writeBoolean(position.snapshotId() != null);
    if (position.snapshotId() != null) {
      out.writeLong(position.snapshotId());
    }
    out.writeBoolean(position.snapshotTimestampMs() != null);
    if (position.snapshotTimestampMs() != null) {
      out.writeLong(position.snapshotTimestampMs());
    }
    byte[] result = out.getCopyOfBuffer();
    out.clear();
    return result;
  }

View on GitHub (pinned to 86d9c8fc54)

Solutions

1. Resume with the same (or newer) Iceberg connector version that wrote the savepoint.
2. If a downgrade is required, start a new job without state instead of restoring incompatible enumerator state.
3. If both versions are known-compatible, allow state rescaling/restart with a compatible serializer or drop operator state via the upgrade mode.
4. Migrate state by writing a custom state-processor job that rewrites the position to the supported version.

Example fix

// before: downgrade then restore
flink-runtime: 1.9.0 (writes version 2) -> restore with 1.6.0 (knows only 1)
// after: restore with matching version
flink savepoint run with 1.6.0 restored using iceberg-flink-runtime 1.6.x

// resume with the exact connector version that wrote the savepoint
String stateVersion = // read version byte from checkpoint metadata if inspectable
if (!expectedConnectorVersion.equals(writingConnectorVersion)) {
  throw new IllegalStateException("Restore requires connector " + writingConnectorVersion);
}

When it happens

Trigger: Restoring a Flink job from a savepoint/checkpoint whose enumerator-state bytes were written by a different (newer) Iceberg connector version that bumped the serializer version.

Common situations: Upgrading the Iceberg flink-runtime jar while resuming from an old savepoint; downgrading the connector and feeding it state written by a newer version.

Understand the failure class

Background: "is not a compatible type" / "cannot merge" errors: when a value's type doesn't match what the library requires — this error's family across 65 libraries.

Related errors

• Unknown version: {version}
• Unknown version:
• Unknown version:
• Unsupported version:
• Unsupported version: {version}

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/b4891adb31b6041b. Report an issue: GitHub.