apache/iceberg · error · IOException

Unknown version:

Error message

Unknown version: 

What it means

IcebergEnumeratorPositionSerializer.deserialize switches on the serialization version; only version 1 is supported. If a checkpoint/serialized byte array carries a different version, it throws IOException('Unknown version: ' + version). This protects against restoring enumerator state written by an incompatible Iceberg version.

Source

Thrown at flink/v2.2/flink/src/main/java/org/apache/iceberg/flink/source/enumerator/IcebergEnumeratorPositionSerializer.java:53

      ThreadLocal.withInitial(() -> new DataOutputSerializer(128));

  @Override
  public int getVersion() {
    return VERSION;
  }

  @Override
  public byte[] serialize(IcebergEnumeratorPosition position) throws IOException {
    return serializeV1(position);
  }

  @Override
  public IcebergEnumeratorPosition deserialize(int version, byte[] serialized) throws IOException {
    switch (version) {
      case 1:
        return deserializeV1(serialized);
      default:
        throw new IOException("Unknown version: " + version);
    }
  }

  private byte[] serializeV1(IcebergEnumeratorPosition position) throws IOException {
    DataOutputSerializer out = SERIALIZER_CACHE.get();
    out.writeBoolean(position.snapshotId() != null);
    if (position.snapshotId() != null) {
      out.writeLong(position.snapshotId());
    }
    out.writeBoolean(position.snapshotTimestampMs() != null);
    if (position.snapshotTimestampMs() != null) {
      out.writeLong(position.snapshotTimestampMs());
    }
    byte[] result = out.getCopyOfBuffer();
    out.clear();
    return result;
  }

View on GitHub (pinned to 86d9c8fc54)

Solutions

1. Resume from a checkpoint/savepoint created with the same (or compatible) Iceberg version as the running jar.
2. Upgrade iceberg-flink so the serializer recognizes the stored version, or downgrade to the previously used version.
3. If the state is unrecoverable, start the streaming job without state (e.g., with StartingStrategy.TABLE_SCAN_THEN_INCREMENTS) to re-plan from the latest snapshot.
4. Check the version byte of stored state to confirm which serializer version produced it before restoring.

Example fix

// before
// restore old savepoint with newer jar: serializer v2 state vs v1 reader -> Unknown version: 2
// after
// align versions, or restart stateless:
ScanContext ctx = ScanContext.builder().streamingStartingStrategy(StreamingStartingStrategy.TABLE_SCAN_THEN_INCREMENTS).build();

// inspect checkpoint metadata for the serializer version before restoring
int v = readEnumeratorStateVersion(checkpointHandle);
if (v != 1) { LOG.warn("State written by serializer version {} is incompatible with this jar", v); }

try {
  position = serializer.deserialize(version, bytes);
} catch (IOException e) {
  if (e.getMessage().startsWith("Unknown version")) {
    // fall back to stateless re-plan from latest snapshot
    position = null;
  } else { throw e; }
}

When it happens

Trigger: Restoring a Flink job from a checkpoint/savepoint whose enumerator state was serialized with a different IcebergEnumeratorPositionSerializer version than the one in the current job jar.

Common situations: Upgrading or downgrading the iceberg-flink version and resuming from an old savepoint; mixing Iceberg jars across job upgrades; hand-editing or replaying serialized enumerator state.

Related errors

• Unknown serialize version:
• Unrecognized version or corrupt state: ${version}
• Unrecognized version or corrupt state: ${version}
• Unknown version:
• Failed to deserialize IcebergSourceSplit. Encountered unsupp

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/59ebebb9eab27f55. Report an issue: GitHub.