apache/iceberg · error · IOException

Unrecognized version or corrupt state: <version>

Error message

Unrecognized version or corrupt state: <version>

What it means

MonitorSource's serializer writes a version byte/long alongside checkpoint state. On restore, if the stored state version is not the current version (e.g. state written by a different Iceberg/Flink release or truncated state), deserialization fails with an IOException indicating unrecognized version or corrupt state.

Source

Thrown at flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/MonitorSource.java:202

          iterator.getClass());

      TableChangeIterator tableChangeIterator = (TableChangeIterator) iterator;
      DataOutputSerializer out = new DataOutputSerializer(8);
      long toStore =
          tableChangeIterator.lastSnapshotId != null ? tableChangeIterator.lastSnapshotId : -1L;
      out.writeLong(toStore);
      return out.getCopyOfBuffer();
    }

    @Override
    public TableChangeIterator deserialize(int version, byte[] serialized) throws IOException {
      if (version == CURRENT_VERSION) {
        DataInputDeserializer in = new DataInputDeserializer(serialized);
        long fromStore = in.readLong();
        return new TableChangeIterator(
            tableLoader, fromStore != -1 ? fromStore : null, maxReadBack);
      } else {
        throw new IOException("Unrecognized version or corrupt state: " + version);
      }
    }
  }
}

View on GitHub (pinned to 86d9c8fc54)

Solutions

1. Use the same (or a compatible) Iceberg version that wrote the savepoint/checkpoint to restore state, then upgrade with state migration if supported.
2. Start a new job without state (drop the savepoint) and let the MonitorSource rebuild its TableChange state.
3. Verify checkpoint files are complete and uncorrupted; re-take the savepoint from a healthy checkpoint.

Example fix

// before
bin/flink run -s old-savepoint-from-iceberg-1.4.jar ...
// after
// upgrade the job graph AND iceberg-flink-runtime to the version that wrote the savepoint first
bin/flink run -s savepoint --jarfile matching-iceberg-version-runtime.jar ...

// Before restoring, check the Iceberg version that wrote the savepoint matches the runtime version

try { restoreFromSavepoint(savepoint); } catch (IOException e) { if (e.getMessage().contains("Unrecognized version")) { startWithoutState(); } else { throw e; } }

When it happens

Trigger: Restoring a Flink job from a savepoint/checkpoint whose operator state for the Iceberg MonitorSource was written by an incompatible Iceberg version, or reading truncated/edited state bytes.

Common situations: Upgrading the Iceberg Flink runtime and resuming from an old savepoint; migrating checkpoints across Iceberg major versions; corrupted checkpoint storage (HDFS/S3 truncation).

Understand the failure class

Background: "is not a compatible type" / "cannot merge" errors: when a value's type doesn't match what the library requires — this error's family across 65 libraries.

Related errors

• Unknown serialize version: ${version}
• Unrecognized version or corrupt state: ${version}
• Unrecognized version or corrupt state: ${version}
• Unrecognized version or corrupt state: ${version}
• Unrecognized version or corrupt state: ${version}

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/1c1773b50874a091. Report an issue: GitHub.