apache/iceberg · error · IOException

Unrecognized version or corrupt state:

Error message

Unrecognized version or corrupt state: 

What it means

TableChangeIterator's deserialize() checks a version byte written by serialize()/snapshot() into the Flink keyed state. If the stored byte array does not carry the current version marker, it assumes the state was produced by an incompatible (older/newer) job or is corrupt, and throws IOException so the state recovery fails fast instead of decoding garbage.

Source

Thrown at flink/v2.3/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/MonitorSource.java:202

          iterator.getClass());

      TableChangeIterator tableChangeIterator = (TableChangeIterator) iterator;
      DataOutputSerializer out = new DataOutputSerializer(8);
      long toStore =
          tableChangeIterator.lastSnapshotId != null ? tableChangeIterator.lastSnapshotId : -1L;
      out.writeLong(toStore);
      return out.getCopyOfBuffer();
    }

    @Override
    public TableChangeIterator deserialize(int version, byte[] serialized) throws IOException {
      if (version == CURRENT_VERSION) {
        DataInputDeserializer in = new DataInputDeserializer(serialized);
        long fromStore = in.readLong();
        return new TableChangeIterator(
            tableLoader, fromStore != -1 ? fromStore : null, maxReadBack);
      } else {
        throw new IOException("Unrecognized version or corrupt state: " + version);
      }
    }
  }
}

View on GitHub (pinned to 86d9c8fc54)

Solutions

1. Take a savepoint with the same Iceberg version used at runtime, or restart the job from scratch (allowNotice/initial state) instead of restoring old state
2. Align the flink/iceberg dependency versions between the job that wrote the state and the one restoring it
3. Inspect the state backend entry to confirm the version byte; if corrupt, delete the offending state entry and reprocess
4. If upgrade is required, drain the maintenance stream (no in-flight state) before upgrading

Example fix

// before: restoring old savepoint with new connector
flink run -s old-savepoint ...  // throws Unrecognized version or corrupt state: 1
// after: use a savepoint/checkpoint taken with matching iceberg-flink-runtime, or start fresh
flink run -s checkpoint-made-with-same-version ...

try {
  iterator = TableChangeIterator.deserialize(...);
} catch (IOException e) {
  LOG.error("State version incompatible; restart without restoring old state", e);
  throw e; // fail the restore; do not silently reprocess with wrong format
}

When it happens

Trigger: Restoring a Flink job from a checkpoint/savepoint created by a different Iceberg/Flink version where the state format version changed; manually modifying or truncating state bytes; reading state serialized by code that wrote no version or a different one.

Common situations: Upgrading the Iceberg Flink runtime while resuming from an old savepoint; replaying a checkpoint across job migrations; state backend corruption after a crash.

Understand the failure class

Background: "Invalid value" and "allowed values are" config errors: what your library rejected and how to fix it — this error's family across 41 libraries.

Related errors

• Unknown version:
• Unknown version:
• Failed to deserialize IcebergSourceSplit. Encountered unsupp
• The Avro schema is not a nullable type: ${schema}
• Fail to serialize at field: %s.

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/06427f1a8b158063. Report an issue: GitHub.