apache/iceberg · error · RuntimeException

Failed to initialize serializerCache for reading data with…

Error message

Failed to initialize serializerCache for reading data with old serializer

What it means

When restoring old serializer state, DynamicRecordInternalSerializer.resolveSchemaCompatibility reflectively invokes the package-private initializeSerializerCache on a DynamicRecordInternalTypeSerializerSnapshot created by an older Iceberg version. If that reflection fails (method renamed, signature changed, class moved), it wraps the cause in this RuntimeException. It exists purely for cross-version state compatibility during restore.

Solutions

1. Resume from a savepoint produced by a compatible Iceberg version (upgrade the savepoint first via a compatible intermediate version if needed)
2. Ensure a single consistent Iceberg Flink jar version is on the classpath (no mixed versions)
3. Restart the job without state if the old state is not required, letting the sink rebuild its serializer cache

// Verify the savepoint's producing Iceberg version is compatible with the runtime version
// before resuming; otherwise start without state.

try {
  restoreFromSavepoint(savepoint);
} catch (RuntimeException e) {
  if (e.getMessage().contains("serializerCache")) {
    // incompatible old serializer state: restart without state or with matching Iceberg version
  }
}

When it happens

Trigger: Restoring a Flink job from a savepoint/checkpoint written by an older Iceberg version whose DynamicRecordInternalTypeSerializerSnapshot internals no longer match the current class (renamed/removed hidden method or changed TableSerializerCache signature).

Common situations: Iceberg version upgrade across a major refactor of the dynamic sink serializers while resuming from an old savepoint; mixed Iceberg jars on the classpath; shading/proguard stripping the hidden method.

Related errors

• Failed to initialize serializerCache for reading data with…
• Unknown read version
• Could not deserialize the WriteResult object
• Could not deserialize the WriteResult object
• Could not deserialize the WriteResult object

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/5b3a008f0c4fd019. Report an issue: GitHub.

Appendix: source

Thrown at flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/sink/dynamic/DynamicRecordInternalSerializer.java:334

    @Override
    public TypeSerializerSchemaCompatibility<DynamicRecordInternal> resolveSchemaCompatibility(
        TypeSerializerSnapshot<DynamicRecordInternal> oldSerializerSnapshot) {
      if (oldSerializerSnapshot.getCurrentVersion() == getCurrentVersion()) {
        return TypeSerializerSchemaCompatibility.compatibleAsIs();
      }

      // Old TypeSerializerSnapshots do not contain the serializer cache, but the newest one does.
      // This will also ensure that we always use the up-to-date cache alongside with its catalog
      // configuration.
      Preconditions.checkNotNull(serializerCache, "serializerCache should not be null");
      try {
        DynMethods.builder("initializeSerializerCache")
            .hiddenImpl(
                DynamicRecordInternalTypeSerializerSnapshot.class, TableSerializerCache.class)
            .build()
            .invoke(oldSerializerSnapshot, serializerCache);
      } catch (Exception e) {
        throw new RuntimeException(
            "Failed to initialize serializerCache for reading data with old serializer", e);
      }

      // This will first read data with the old serializer, then switch to the most recent one.
      return TypeSerializerSchemaCompatibility.compatibleAfterMigration();
    }

    @Override
    public TypeSerializer<DynamicRecordInternal> restoreSerializer() {
      if (getCurrentVersion() < MOST_RECENT_VERSION) {
        // If this serializer is not the most recent one, we need to read old data with the correct
        // parameters.
        return new DynamicRecordInternalSerializer(serializerCache, writeSchemaAndSpec, false);
      }

      // In all other cases, we just use the newest serializer.
      return new DynamicRecordInternalSerializer(serializerCache, writeSchemaAndSpec, true);
    }

View on GitHub (pinned to 86d9c8fc54)