apache/iceberg · error · IOException

Unrecognized version or corrupt state: ${version}

Error message

Unrecognized version or corrupt state: ${version}

What it means

DynamicCommittableSerializer.deserialize reads a version byte and only knows VERSION_1 and VERSION_2; anything else throws this IOException. It guards against decoding committables written by newer or incompatible Iceberg/Flink sink versions, treating unknown versions as corrupt state.

Source

Thrown at flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/sink/dynamic/DynamicCommittableSerializer.java:70

    view.writeInt(numManifests);
    for (int i = 0; i < numManifests; i++) {
      byte[] manifest = committable.manifests()[i];
      view.writeInt(manifest.length);
      view.write(manifest);
    }

    return out.toByteArray();
  }

  @Override
  public DynamicCommittable deserialize(int version, byte[] serialized) throws IOException {
    if (version == VERSION_1) {
      return deserializeV1(serialized);
    } else if (version == VERSION_2) {
      return deserializeV2(serialized);
    }

    throw new IOException("Unrecognized version or corrupt state: " + version);
  }

  private DynamicCommittable deserializeV1(byte[] serialized) throws IOException {
    DataInputDeserializer view = new DataInputDeserializer(serialized);
    WriteTarget key = WriteTarget.deserializeFrom(view);
    String jobId = view.readUTF();
    String operatorId = view.readUTF();
    long checkpointId = view.readLong();
    int manifestLen = view.readInt();
    byte[] manifestBuf = new byte[manifestLen];
    view.read(manifestBuf);
    return new DynamicCommittable(
        new TableKey(key.tableName(), key.branch()),
        new byte[][] {manifestBuf},
        jobId,
        operatorId,
        checkpointId);
  }

View on GitHub (pinned to 86d9c8fc54)

Solutions

1. Use an Iceberg version that understands the committable version byte in the savepoint (match the writer version to the state)
2. Recreate the sink without restoring from the incompatible savepoint and let the sink resume from committed snapshots (max committed checkpoint id)
3. Verify no data corruption: compare the savepoint contents against the expected serializer versions of both the producing and consuming Iceberg versions

// Check that the Iceberg versions producing and consuming the savepoint match
// (compare iceberg-flink-runtime versions of the old and new job).

try {
  committable = serializer.deserialize(version, bytes);
} catch (IOException e) {
  // unknown version: cannot recover by retry; restore with matching Iceberg version or drop state
}

When it happens

Trigger: Restoring a Flink savepoint/checkpoint whose committable state was written by a different (newer or older) Iceberg sink version with an unknown serialization version byte.

Common situations: Upgrading the Iceberg Flink sink version mid-stream and resuming from an old savepoint; mixing job versions on the same checkpoint state; manually editing or corrupting checkpoint bytes.

Related errors

• Unknown serialize version: ${version}
• Unrecognized version or corrupt state: ${version}
• Failed to deserialize IcebergSourceSplit. Encountered unsupp
• Unrecognized version or corrupt state: <version>
• Unrecognized version or corrupt state: ${version}

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/19fbbc8873d278b3. Report an issue: GitHub.