apache/iceberg · error · RuntimeException

Failed to initialize serializerCache for reading data with…

Error message

Failed to initialize serializerCache for reading data with old serializer

What it means

When restoring state written by the old (pre-cache) DynamicRecordInternal serializer snapshot, resolveSchemaCompatibility reflectively calls initializeSerializerCache on the old snapshot. If that reflective invocation fails for any reason, the exception is wrapped in a RuntimeException so migration cannot proceed.

Solutions

1. Ensure the Iceberg version being restored into is compatible with the version that wrote the state
2. Re-write the checkpoint/savepoint with the new version and restart from it
3. Inspect the wrapped cause to fix the underlying initialization failure

// Verify state was written by a compatible version before restore
assert savepointIcebergVersion <= currentIcebergVersion;

try {
  compat = snapshot.resolveSchemaCompatibility(oldSerializer);
} catch (RuntimeException e) {
  throw new StateMigrationException("Cannot migrate old serializer state", e.getCause());
}

When it happens

Trigger: Restoring a Flink job from a savepoint/checkpoint written by an old DynamicRecordInternalTypeSerializer version, where initializeSerializerCache throws (e.g. missing TableSerializerCache state or signature change across versions).

Common situations: Upgrading Iceberg Flink sink across versions with old operator state; refactoring of DynamicRecordInternalTypeSerializerSnapshot internals between releases breaking the hidden-method call.

Understand the failure class

Background: "This is a bug, please report it": internal invariant violations, unreachable panics, and SNH errors explained — this error's family across 47 libraries.

Related errors

• Unknown read version:
• Unknown read version
• Unknown serialize version:
• Unrecognized version or corrupt state:
• Could not deserialize the WriteResult object

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/b4e81ceea6920236. Report an issue: GitHub.

Appendix: source

Thrown at flink/v2.3/flink/src/main/java/org/apache/iceberg/flink/sink/dynamic/DynamicRecordInternalSerializer.java:334

    @Override
    public TypeSerializerSchemaCompatibility<DynamicRecordInternal> resolveSchemaCompatibility(
        TypeSerializerSnapshot<DynamicRecordInternal> oldSerializerSnapshot) {
      if (oldSerializerSnapshot.getCurrentVersion() == getCurrentVersion()) {
        return TypeSerializerSchemaCompatibility.compatibleAsIs();
      }

      // Old TypeSerializerSnapshots do not contain the serializer cache, but the newest one does.
      // This will also ensure that we always use the up-to-date cache alongside with its catalog
      // configuration.
      Preconditions.checkNotNull(serializerCache, "serializerCache should not be null");
      try {
        DynMethods.builder("initializeSerializerCache")
            .hiddenImpl(
                DynamicRecordInternalTypeSerializerSnapshot.class, TableSerializerCache.class)
            .build()
            .invoke(oldSerializerSnapshot, serializerCache);
      } catch (Exception e) {
        throw new RuntimeException(
            "Failed to initialize serializerCache for reading data with old serializer", e);
      }

      // This will first read data with the old serializer, then switch to the most recent one.
      return TypeSerializerSchemaCompatibility.compatibleAfterMigration();
    }

    @Override
    public TypeSerializer<DynamicRecordInternal> restoreSerializer() {
      if (getCurrentVersion() < MOST_RECENT_VERSION) {
        // If this serializer is not the most recent one, we need to read old data with the correct
        // parameters.
        return new DynamicRecordInternalSerializer(serializerCache, writeSchemaAndSpec, false);
      }

      // In all other cases, we just use the newest serializer.
      return new DynamicRecordInternalSerializer(serializerCache, writeSchemaAndSpec, true);
    }

View on GitHub (pinned to 86d9c8fc54)