{"record":{"id":"2b751a8cd043015f","repo":"apache/iceberg","slug":"failed-to-initialize-serializercache-for-reading-d-2b751a","errorCode":null,"errorMessage":"Failed to initialize serializerCache for reading data with old serializer","messagePattern":"Failed to initialize serializerCache for reading data with old serializer","errorType":"exception","errorClass":"RuntimeException","httpStatus":null,"severity":"error","filePath":"flink/v2.1/flink/src/main/java/org/apache/iceberg/flink/sink/dynamic/DynamicRecordInternalSerializer.java","lineNumber":334,"sourceCode":"    @Override\n    public TypeSerializerSchemaCompatibility<DynamicRecordInternal> resolveSchemaCompatibility(\n        TypeSerializerSnapshot<DynamicRecordInternal> oldSerializerSnapshot) {\n      if (oldSerializerSnapshot.getCurrentVersion() == getCurrentVersion()) {\n        return TypeSerializerSchemaCompatibility.compatibleAsIs();\n      }\n\n      // Old TypeSerializerSnapshots do not contain the serializer cache, but the newest one does.\n      // This will also ensure that we always use the up-to-date cache alongside with its catalog\n      // configuration.\n      Preconditions.checkNotNull(serializerCache, \"serializerCache should not be null\");\n      try {\n        DynMethods.builder(\"initializeSerializerCache\")\n            .hiddenImpl(\n                DynamicRecordInternalTypeSerializerSnapshot.class, TableSerializerCache.class)\n            .build()\n            .invoke(oldSerializerSnapshot, serializerCache);\n      } catch (Exception e) {\n        throw new RuntimeException(\n            \"Failed to initialize serializerCache for reading data with old serializer\", e);\n      }\n\n      // This will first read data with the old serializer, then switch to the most recent one.\n      return TypeSerializerSchemaCompatibility.compatibleAfterMigration();\n    }\n\n    @Override\n    public TypeSerializer<DynamicRecordInternal> restoreSerializer() {\n      if (getCurrentVersion() < MOST_RECENT_VERSION) {\n        // If this serializer is not the most recent one, we need to read old data with the correct\n        // parameters.\n        return new DynamicRecordInternalSerializer(serializerCache, writeSchemaAndSpec, false);\n      }\n\n      // In all other cases, we just use the newest serializer.\n      return new DynamicRecordInternalSerializer(serializerCache, writeSchemaAndSpec, true);\n    }","sourceCodeStart":316,"sourceCodeEnd":352,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/flink/v2.1/flink/src/main/java/org/apache/iceberg/flink/sink/dynamic/DynamicRecordInternalSerializer.java#L316-L352","documentation":"During schema-compatibility resolution, DynamicRecordInternalSerializer may restore an OLD serializer snapshot (pre serializerCache). It initializes the cache reflectively via DynMethods on the hidden snapshot class; any failure there is wrapped in a RuntimeException with this message, aborting restore from old-version state.","triggerScenarios":"Restoring a Flink job from a savepoint/checkpoint written by an older Iceberg version whose DynamicRecordInternalTypeSerializerSnapshot lacks the expected initializeSerializerCache method or TableSerializerCache class.","commonSituations":"Iceberg runtime upgrades across restore with dynamic-sink operator state; mismatched jar versions where the hidden method signature changed; reflective access blocked by JPMS/module settings.","solutions":["Restore using the exact Iceberg version that wrote the savepoint, drain, then upgrade on a clean start.","Ensure the iceberg-flink runtime version is consistent across all nodes and matches the code performing the restore.","If reflective access fails due to module restrictions, add the required --add-opens for org.apache.iceberg.flink.sink.dynamic.","Avoid resuming in-flight dynamic-sink state across major serializer changes; restart the pipeline without the old snapshot."],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"try {\n  compatibility = serializerSnapshot.resolveSchemaCompatibility(newSerializer);\n} catch (RuntimeException e) {\n  if (e.getMessage().contains(\"Failed to initialize serializerCache\")) {\n    // restore with the original Iceberg version, or restart without the old snapshot\n  }\n  throw e;\n}","preventionTips":["Match the Iceberg runtime version to the one that wrote the savepoint when restoring.","Avoid reserving in-flight dynamic-sink state across breaking serializer upgrades.","Add --add-opens flags if reflective access is restricted in the deployment environment."],"tags":["flink","dynamic-sink","serialization","restore"],"backgroundTag":"module-init-failed","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}