apache/iceberg · error · IOException

Could not deserialize the WriteResult object

Error message

Could not deserialize the WriteResult object

What it means

WriteResultSerializer.deserialize reads an embedded Java-serialized WriteResult buffer and uses InstantiationUtil.deserializeObject; a ClassNotFoundException during that deserialization is rethrown as IOException 'Could not deserialize the WriteResult object'. The bytes are valid, but the class required to decode them is not on the deserializing side's classloader.

Source

Thrown at flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/sink/WriteResultSerializer.java:58

  public byte[] serialize(WriteResult writeResult) throws IOException {
    ByteArrayOutputStream out = new ByteArrayOutputStream();
    DataOutputViewStreamWrapper view = new DataOutputViewStreamWrapper(out);
    byte[] result = InstantiationUtil.serializeObject(writeResult);
    view.write(result);
    return out.toByteArray();
  }

  @Override
  public WriteResult deserialize(int version, byte[] serialized) throws IOException {
    if (version == 1) {
      DataInputDeserializer view = new DataInputDeserializer(serialized);
      byte[] resultBuf = new byte[serialized.length];
      view.read(resultBuf);
      try {
        return InstantiationUtil.deserializeObject(
            resultBuf, IcebergCommittableSerializer.class.getClassLoader());
      } catch (ClassNotFoundException cnc) {
        throw new IOException("Could not deserialize the WriteResult object", cnc);
      }
    }
    throw new IOException("Unrecognized version or corrupt state: " + version);
  }
}

View on GitHub (pinned to 86d9c8fc54)

Solutions

1. Ensure the iceberg-flink-runtime jar is packaged in the user job jar (not just on the cluster lib path) so the same classloader can resolve WriteResult
2. Check for classloader conflicts (duplicate or relocated org.apache.iceberg packages) and use user-code-header/unresolved-classloader settings appropriately
3. Restore savepoints with the same Iceberg version that produced them
4. Flink 1.19+: try enabling classloader.resolve-order or the compat mode for serializers (execution.checkpointing.unaligned / pipeline.classloader defaults) to keep deserialization on the user classloader

Example fix

// before
# iceberg-flink-runtime only in $FLINK_HOME/lib, user jar compiled against different version -> CNFE
// after
mvn shade: put iceberg-flink-runtime-1.20-*.jar into the job jar's dependencies
# or align versions:
# $FLINK_HOME/lib and job jar both use iceberg-flink-runtime-1.20.0.jar

// ensure WriteResult is loadable by the current classloader
Class.forName("org.apache.iceberg.flink.sink.WriteResult");

boolean canDeserializeWriteResult() {
  try {
    Class.forName("org.apache.iceberg.flink.sink.WriteResult", true,
        Thread.currentThread().getContextClassLoader());
    return true;
  } catch (ClassNotFoundException e) {
    return false;
  }
}

try {
  return InstantiationUtil.deserializeObject(buf, loader);
} catch (IOException e) {
  if (e.getCause() instanceof ClassNotFoundException) {
    throw new IllegalStateException("Iceberg runtime jar missing from user classloader", e);
  }
  throw e;
}

When it happens

Trigger: Transferring WriteResult committables between Writer/Agregator/Committer operators (or across savepoint restore) when org.apache.iceberg.flink.sink.WriteResult and its nested classes are missing from the user classloader — e.g. user-code classloader lacks the Iceberg jars, or parent-first/child-first classloading isolates the class.

Common situations: Running Flink with classloader.check-leaked-classloader or child-first loading where Iceberg classes live in the user jar but the serializer is loaded by a different classloader; savepoint from a different Iceberg version whose WriteResult layout/class moved; shaded/relocated Iceberg packages mismatch.

Related errors

• Unrecognized version or corrupt state: ${version}
• Unrecognized version or corrupt state: ${version}
• Failed to deserialize the split.
• Failed to deserialize the split.
• Could not deserialize the WriteResult object

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/72e95f8589409ad7. Report an issue: GitHub.