apache/iceberg · error · IOException

Unrecognized version or corrupt state: ${version}

Error message

Unrecognized version or corrupt state: ${version}

What it means

WriteResultSerializer.deserialize only supports its current VERSION marker; if the leading version read from the serialized bytes does not match, it throws IOException 'Unrecognized version or corrupt state: <version>'. This protects against decoding committables written by an incompatible serializer version or from a corrupted stream, mirroring IcebergCommittableSerializer's guard.

Source

Thrown at flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/sink/WriteResultSerializer.java:61

    byte[] result = InstantiationUtil.serializeObject(writeResult);
    view.write(result);
    return out.toByteArray();
  }

  @Override
  public WriteResult deserialize(int version, byte[] serialized) throws IOException {
    if (version == 1) {
      DataInputDeserializer view = new DataInputDeserializer(serialized);
      byte[] resultBuf = new byte[serialized.length];
      view.read(resultBuf);
      try {
        return InstantiationUtil.deserializeObject(
            resultBuf, IcebergCommittableSerializer.class.getClassLoader());
      } catch (ClassNotFoundException cnc) {
        throw new IOException("Could not deserialize the WriteResult object", cnc);
      }
    }
    throw new IOException("Unrecognized version or corrupt state: " + version);
  }
}

View on GitHub (pinned to 86d9c8fc54)

Solutions

1. Resume with the iceberg-flink-runtime version that wrote the checkpoint/savepoint
2. If versions must change, drain the pipeline first (commit all in-flight checkpoints) and restart without old committable state, or use --allowNonRestoredState if safe
3. Verify single consistent Iceberg jar version on the classpath (no duplicates/shadowing)
4. If corruption is suspected, inspect the failing checkpoint and restart from an earlier valid one

Example fix

// before
flink run -s savepoint-written-by-iceberg-1.4 --jarfile iceberg-flink-runtime-1.20.0.jar job.jar
// after
flink run -s savepoint-written-by-iceberg-1.4 --jarfile iceberg-flink-runtime-1.4.3.jar job.jar
# or: fully drain + fresh restart before upgrading the runtime

try {
  return serializer.deserialize(version, bytes);
} catch (IOException e) {
  if (e.getMessage().startsWith("Unrecognized version")) {
    // incompatible state or corruption: never retry blindly; restart from valid checkpoint
    throw new StateIncompatibleException("WriteResult format mismatch: " + e.getMessage(), e);
  }
  throw e;
}

When it happens

Trigger: Restoring from a savepoint/checkpoint written by a different Iceberg version with a different WriteResultSerializer layout, or deserializing truncated/garbled bytes between Writer and Aggregator/Committer operators so the version marker reads as an unexpected value.

Common situations: Upgrading iceberg-flink-runtime while resuming from an old savepoint; network/serialization corruption within a checkpoint (rare); classpath mixing two serializer implementations with different VERSION constants.

Understand the failure class

Background: "Invalid ... format", "must be in format X", "does not look like a ..." — invalid argument format errors across CLI tools and libraries — this error's family across 17 libraries.

Related errors

• Unrecognized version or corrupt state: ${version}
• Unrecognized version or corrupt state: <version>
• Unknown serialize version: ${version}
• Could not deserialize the WriteResult object
• Unrecognized version or corrupt state: ${version}

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/a1e5b362209fa120. Report an issue: GitHub.