apache/iceberg · error · IOException

Unrecognized version or corrupt state: ${version}

Error message

Unrecognized version or corrupt state: ${version}

What it means

DynamicWriteResultSerializer.deserialize only recognizes specific serialized version bytes for DynamicWriteResult records. When the leading version byte is not one of the known versions, the serializer cannot know how to decode the payload and throws this IOException. It almost always indicates the byte stream was not produced by a compatible version of this serializer.

Source

Thrown at flink/v2.1/flink/src/main/java/org/apache/iceberg/flink/sink/dynamic/DynamicWriteResultSerializer.java:62

    view.writeInt(writeResult.specId());
    byte[] result = WRITE_RESULT_SERIALIZER.serialize(writeResult.writeResult());
    view.write(result);
    return out.toByteArray();
  }

  @Override
  public DynamicWriteResult deserialize(int version, byte[] serialized) throws IOException {
    if (version == 1) {
      DataInputDeserializer view = new DataInputDeserializer(serialized);
      TableKey key = TableKey.deserializeFrom(view);
      int specId = view.readInt();
      byte[] resultBuf = new byte[view.available()];
      view.read(resultBuf);
      WriteResult writeResult = WRITE_RESULT_SERIALIZER.deserialize(version, resultBuf);
      return new DynamicWriteResult(key, specId, writeResult);
    }

    throw new IOException("Unrecognized version or corrupt state: " + version);
  }
}

View on GitHub (pinned to 86d9c8fc54)

Solutions

1. Use the same Iceberg Flink connector version that produced the serialized data when restoring from a checkpoint/savepoint.
2. Regenerate the data instead of deserializing stale serialized state; drop old checkpoints if data is transient.
3. Check for byte corruption or misaligned stream position before the version byte is read.

Example fix

// before: resuming a job from a checkpoint made with iceberg-flink-runtime 1.9.x using 1.8.x jar
// after: align runtime version or discard the incompatible checkpoint
// job: --jar iceberg-flink-runtime-1.20-1.9.2.jar  (same version that wrote the checkpoint)

// verify serialized bytes come from the same connector version
assert icebergFlinkRuntimeVersion.equals(PRODUCING_RUNTIME_VERSION);

try {
  DynamicWriteResult r = serializer.deserialize(in);
} catch (IOException e) {
  if (e.getMessage().contains("Unrecognized version or corrupt state")) {
    throw new IllegalStateException("Incompatible serialized state; restore from a matching connector version checkpoint", e);
  }
  throw e;
}

When it happens

Trigger: Deserializing a DynamicWriteResult whose data bytes were written by a newer or older serializer version, feeding corrupt/truncated bytes to DataInputViewStreamSerializer, or deserializing checkpoints/state written by a different Iceberg Flink version.

Common situations: Upgrading or downgrading the Iceberg Flink connector between jobs while resuming from savepoints/checkpoints, or manually replaying serialized records across incompatible versions.

Understand the failure class

Background: Invalid enum value errors: "Unknown type", "Invalid scope", "must be one of" — when a string is not on the library's allowed list — this error's family across 23 libraries.

Related errors

• Failed to deserialize IcebergSourceSplit. Encountered unsupp
• Unknown read version: ${readVersion}
• Failed to deserialize IcebergSourceSplit. Encountered unsupp
• Unknown read version:
• Unrecognized version or corrupt state:

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/f83485e9c06b90d6. Report an issue: GitHub.