apache/iceberg · error · RuntimeException

Failed to deserialize the split.

Error message

Failed to deserialize the split.

What it means

IcebergSourceSplit.deserializeV1 uses Flink's Java-serialization fallback (InstantiationUtil.deserializeObject). If the bytes reference a class not on the classpath, ClassNotFoundException is wrapped in this RuntimeException.

Source

Thrown at flink/v2.2/flink/src/main/java/org/apache/iceberg/flink/source/split/IcebergSourceSplit.java:131

                        .add("length", fileScanTask.length())
                        .toString())
            .collect(Collectors.toList()));
  }

  byte[] serializeV1() throws IOException {
    if (serializedBytesCache == null) {
      serializedBytesCache = InstantiationUtil.serializeObject(this);
    }

    return serializedBytesCache;
  }

  static IcebergSourceSplit deserializeV1(byte[] serialized) throws IOException {
    try {
      return InstantiationUtil.deserializeObject(
          serialized, IcebergSourceSplit.class.getClassLoader());
    } catch (ClassNotFoundException e) {
      throw new RuntimeException("Failed to deserialize the split.", e);
    }
  }

  byte[] serializeV2() throws IOException {
    return serialize(2);
  }

  byte[] serializeV3() throws IOException {
    return serialize(3);
  }

  private byte[] serialize(int version) throws IOException {
    if (serializedBytesCache == null) {
      DataOutputSerializer out = SERIALIZER_CACHE.get();
      Collection<FileScanTask> fileScanTasks = task.tasks();
      Preconditions.checkArgument(
          fileOffset >= 0 && fileOffset < fileScanTasks.size(),
          "Invalid file offset: %s. Should be within the range of [0, %s)",

View on GitHub (pinned to 86d9c8fc54)

Solutions

1. Put the full iceberg-flink-runtime jar on the classpath consistently across JM and TM
2. Fix Flink classloading config (e.g. classloader.check-leaked-classloader, resolve-order) so user classes resolve
3. Ensure the same Iceberg version is used to serialize and deserialize
4. Prefer V2/V3 split serialization (table/scan JSON) which avoids Java serialization
5. If restoring old savepoints, ensure legacy classes referenced by V1 payloads are present

Example fix

// before: split jar with only iceberg-flink-runtime but core classes excluded
// after: use full shaded runtime
// org.apache.iceberg:iceberg-flink-runtime-1.19 (shaded, includes core)

// ensure the runtime jar is complete before reading state
Class.forName("org.apache.iceberg.flink.source.split.IcebergSourceSplit");

try {
  return IcebergSourceSplit.deserializeV1(bytes);
} catch (RuntimeException e) {
  if (e.getCause() instanceof ClassNotFoundException) {
    throw new IllegalStateException("Iceberg classes missing from task classpath; check iceberg-flink-runtime jar", e);
  }
  throw e;
}

When it happens

Trigger: Deserializing a V1-encoded IcebergSourceSplit whose payload classes (e.g. TableSerializations, internal types) are missing from the task classloader.

Common situations: Shaded vs non-shaded connector mismatch on the Flink classpath; user-classloader isolation (parent-first/child-first) hiding Iceberg classes; mixed Iceberg versions between job and taskmanager classpath.

Related errors

• Failed to decode partition
• Could not deserialize the WriteResult object
• Failed to deserialize sort key sketch
• Fail to deserialize data statistics
• Fail to deserialize aggregated statistics,change to v1

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/c9a13c33ff1dcb56. Report an issue: GitHub.