apache/iceberg · error · IOException

Unrecognized version or corrupt state:

Error message

Unrecognized version or corrupt state: 

What it means

TableChangeIterator's serializer in MonitorSource stores a single long (last snapshot id) tagged with a version byte. On deserialize, if the stored state's version is not CURRENT_VERSION, the state is neither readable nor migratable, so an IOException 'Unrecognized version or corrupt state' is thrown.

Source

Thrown at flink/v2.1/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/MonitorSource.java:202

          iterator.getClass());

      TableChangeIterator tableChangeIterator = (TableChangeIterator) iterator;
      DataOutputSerializer out = new DataOutputSerializer(8);
      long toStore =
          tableChangeIterator.lastSnapshotId != null ? tableChangeIterator.lastSnapshotId : -1L;
      out.writeLong(toStore);
      return out.getCopyOfBuffer();
    }

    @Override
    public TableChangeIterator deserialize(int version, byte[] serialized) throws IOException {
      if (version == CURRENT_VERSION) {
        DataInputDeserializer in = new DataInputDeserializer(serialized);
        long fromStore = in.readLong();
        return new TableChangeIterator(
            tableLoader, fromStore != -1 ? fromStore : null, maxReadBack);
      } else {
        throw new IOException("Unrecognized version or corrupt state: " + version);
      }
    }
  }
}

View on GitHub (pinned to 86d9c8fc54)

Solutions

1. Resume from a checkpoint/savepoint produced by the same (or compatible) Iceberg version that defines the same CURRENT_VERSION.
2. If the version change is intentional, allow the job to start fresh (drop the source state) — the monitor source will rescan from the current snapshot, possibly re-processing recent changes.
3. Verify checkpoint bytes are intact (no truncation in the state backend storage) before restoring.

Example fix

// before
// restoring savepoint from Iceberg 1.4 state into a build with CURRENT_VERSION=2 -> IOException
// after
// start the job without the old savepoint, or upgrade to the version that wrote it
flink run -s <compatible-savepoint> ...

try {
  iterator = serializer.deserialize(version, bytes);
} catch (IOException e) {
  LOG.warn("Incompatible monitor-source state ({}); starting from current snapshot", e.getMessage());
  iterator = new TableChangeIterator(tableLoader, null, maxReadBack);
}

When it happens

Trigger: Restoring a Flink source whose serializer state was written by an older Iceberg version (different CURRENT_VERSION) or whose state bytes are truncated/corrupt in the checkpoint/savepoint.

Common situations: Upgrading Iceberg/Flink across a state-format version change while resuming from an old savepoint; corrupted checkpoint storage (e.g. object store truncation); manually edited or migrated operator state.

Understand the failure class

Background: "failed to unmarshal" / json.Unmarshal errors: why parsing a response into a Go struct fails and how to fix it — this error's family across 23 libraries.

Related errors

• Failed to deserialize IcebergSourceSplit. Encountered unsupp
• Unrecognized version or corrupt state: <version>
• Unknown serialize version: ${version}
• Unrecognized version or corrupt state: ${version}
• Failed to close equality delta writer

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/b35ed995619f8684. Report an issue: GitHub.