{"record":{"id":"b4e81ceea6920236","repo":"apache/iceberg","slug":"failed-to-initialize-serializercache-for-reading-d-b4e81c","errorCode":null,"errorMessage":"Failed to initialize serializerCache for reading data with old serializer","messagePattern":"Failed to initialize serializerCache for reading data with old serializer","errorType":"exception","errorClass":"RuntimeException","httpStatus":null,"severity":"error","filePath":"flink/v2.3/flink/src/main/java/org/apache/iceberg/flink/sink/dynamic/DynamicRecordInternalSerializer.java","lineNumber":334,"sourceCode":"    @Override\n    public TypeSerializerSchemaCompatibility<DynamicRecordInternal> resolveSchemaCompatibility(\n        TypeSerializerSnapshot<DynamicRecordInternal> oldSerializerSnapshot) {\n      if (oldSerializerSnapshot.getCurrentVersion() == getCurrentVersion()) {\n        return TypeSerializerSchemaCompatibility.compatibleAsIs();\n      }\n\n      // Old TypeSerializerSnapshots do not contain the serializer cache, but the newest one does.\n      // This will also ensure that we always use the up-to-date cache alongside with its catalog\n      // configuration.\n      Preconditions.checkNotNull(serializerCache, \"serializerCache should not be null\");\n      try {\n        DynMethods.builder(\"initializeSerializerCache\")\n            .hiddenImpl(\n                DynamicRecordInternalTypeSerializerSnapshot.class, TableSerializerCache.class)\n            .build()\n            .invoke(oldSerializerSnapshot, serializerCache);\n      } catch (Exception e) {\n        throw new RuntimeException(\n            \"Failed to initialize serializerCache for reading data with old serializer\", e);\n      }\n\n      // This will first read data with the old serializer, then switch to the most recent one.\n      return TypeSerializerSchemaCompatibility.compatibleAfterMigration();\n    }\n\n    @Override\n    public TypeSerializer<DynamicRecordInternal> restoreSerializer() {\n      if (getCurrentVersion() < MOST_RECENT_VERSION) {\n        // If this serializer is not the most recent one, we need to read old data with the correct\n        // parameters.\n        return new DynamicRecordInternalSerializer(serializerCache, writeSchemaAndSpec, false);\n      }\n\n      // In all other cases, we just use the newest serializer.\n      return new DynamicRecordInternalSerializer(serializerCache, writeSchemaAndSpec, true);\n    }","sourceCodeStart":316,"sourceCodeEnd":352,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/flink/v2.3/flink/src/main/java/org/apache/iceberg/flink/sink/dynamic/DynamicRecordInternalSerializer.java#L316-L352","documentation":"When restoring state written by the old (pre-cache) DynamicRecordInternal serializer snapshot, resolveSchemaCompatibility reflectively calls initializeSerializerCache on the old snapshot. If that reflective invocation fails for any reason, the exception is wrapped in a RuntimeException so migration cannot proceed.","triggerScenarios":"Restoring a Flink job from a savepoint/checkpoint written by an old DynamicRecordInternalTypeSerializer version, where initializeSerializerCache throws (e.g. missing TableSerializerCache state or signature change across versions).","commonSituations":"Upgrading Iceberg Flink sink across versions with old operator state; refactoring of DynamicRecordInternalTypeSerializerSnapshot internals between releases breaking the hidden-method call.","solutions":["Ensure the Iceberg version being restored into is compatible with the version that wrote the state","Re-write the checkpoint/savepoint with the new version and restart from it","Inspect the wrapped cause to fix the underlying initialization failure"],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":"// Verify state was written by a compatible version before restore\nassert savepointIcebergVersion <= currentIcebergVersion;","typeGuard":null,"tryCatchPattern":"try {\n  compat = snapshot.resolveSchemaCompatibility(oldSerializer);\n} catch (RuntimeException e) {\n  throw new StateMigrationException(\"Cannot migrate old serializer state\", e.getCause());\n}","preventionTips":["Upgrade through intermediate Iceberg versions rather than jumping many releases","Test savepoint restore in staging before production upgrades","Keep the hidden initializeSerializerCache signature intact when modifying the snapshot class"],"tags":["flink","serialization","state-restore"],"backgroundTag":"internal-invariant-violation","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}