{"record":{"id":"5b3a008f0c4fd019","repo":"apache/iceberg","slug":"failed-to-initialize-serializercache-for-reading-d","errorCode":null,"errorMessage":"Failed to initialize serializerCache for reading data with old serializer","messagePattern":"Failed to initialize serializerCache for reading data with old serializer","errorType":"exception","errorClass":"RuntimeException","httpStatus":null,"severity":"error","filePath":"flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/sink/dynamic/DynamicRecordInternalSerializer.java","lineNumber":334,"sourceCode":"    @Override\n    public TypeSerializerSchemaCompatibility<DynamicRecordInternal> resolveSchemaCompatibility(\n        TypeSerializerSnapshot<DynamicRecordInternal> oldSerializerSnapshot) {\n      if (oldSerializerSnapshot.getCurrentVersion() == getCurrentVersion()) {\n        return TypeSerializerSchemaCompatibility.compatibleAsIs();\n      }\n\n      // Old TypeSerializerSnapshots do not contain the serializer cache, but the newest one does.\n      // This will also ensure that we always use the up-to-date cache alongside with its catalog\n      // configuration.\n      Preconditions.checkNotNull(serializerCache, \"serializerCache should not be null\");\n      try {\n        DynMethods.builder(\"initializeSerializerCache\")\n            .hiddenImpl(\n                DynamicRecordInternalTypeSerializerSnapshot.class, TableSerializerCache.class)\n            .build()\n            .invoke(oldSerializerSnapshot, serializerCache);\n      } catch (Exception e) {\n        throw new RuntimeException(\n            \"Failed to initialize serializerCache for reading data with old serializer\", e);\n      }\n\n      // This will first read data with the old serializer, then switch to the most recent one.\n      return TypeSerializerSchemaCompatibility.compatibleAfterMigration();\n    }\n\n    @Override\n    public TypeSerializer<DynamicRecordInternal> restoreSerializer() {\n      if (getCurrentVersion() < MOST_RECENT_VERSION) {\n        // If this serializer is not the most recent one, we need to read old data with the correct\n        // parameters.\n        return new DynamicRecordInternalSerializer(serializerCache, writeSchemaAndSpec, false);\n      }\n\n      // In all other cases, we just use the newest serializer.\n      return new DynamicRecordInternalSerializer(serializerCache, writeSchemaAndSpec, true);\n    }","sourceCodeStart":316,"sourceCodeEnd":352,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/sink/dynamic/DynamicRecordInternalSerializer.java#L316-L352","documentation":"When restoring old serializer state, DynamicRecordInternalSerializer.resolveSchemaCompatibility reflectively invokes the package-private initializeSerializerCache on a DynamicRecordInternalTypeSerializerSnapshot created by an older Iceberg version. If that reflection fails (method renamed, signature changed, class moved), it wraps the cause in this RuntimeException. It exists purely for cross-version state compatibility during restore.","triggerScenarios":"Restoring a Flink job from a savepoint/checkpoint written by an older Iceberg version whose DynamicRecordInternalTypeSerializerSnapshot internals no longer match the current class (renamed/removed hidden method or changed TableSerializerCache signature).","commonSituations":"Iceberg version upgrade across a major refactor of the dynamic sink serializers while resuming from an old savepoint; mixed Iceberg jars on the classpath; shading/proguard stripping the hidden method.","solutions":["Resume from a savepoint produced by a compatible Iceberg version (upgrade the savepoint first via a compatible intermediate version if needed)","Ensure a single consistent Iceberg Flink jar version is on the classpath (no mixed versions)","Restart the job without state if the old state is not required, letting the sink rebuild its serializer cache"],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":"// Verify the savepoint's producing Iceberg version is compatible with the runtime version\n// before resuming; otherwise start without state.","typeGuard":null,"tryCatchPattern":"try {\n  restoreFromSavepoint(savepoint);\n} catch (RuntimeException e) {\n  if (e.getMessage().contains(\"serializerCache\")) {\n    // incompatible old serializer state: restart without state or with matching Iceberg version\n  }\n}","preventionTips":["Upgrade through the documented migration path instead of jumping many Iceberg versions with old state","Keep one iceberg-flink version on all nodes","Test savepoint restore compatibility in staging before upgrading"],"tags":["flink","serialization","savepoint","compatibility"],"backgroundTag":"class-not-found","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}