{"record":{"id":"40963249f7d1a2e5","repo":"zhisheng17/flink-learning","slug":"failed-to-deserialize-state-handle-from-configmap","errorCode":null,"errorMessage":"Failed to deserialize state handle from ConfigMap data {}.","messagePattern":"Failed to deserialize state handle from ConfigMap data (.+?)\\.","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"flink-learning-k8s/flink-k8s/src/main/java/org/apache/flink/kubernetes/highavailability/KubernetesStateHandleStore.java","lineNumber":425,"sourceCode":"\t@Override\n\tpublic void releaseAll() {\n\t\t// noop\n\t}\n\n\t@Override\n\tpublic String toString() {\n\t\treturn this.getClass().getSimpleName() + \"{configMapName='\" + configMapName + \"'}\";\n\t}\n\n\tprivate RetrievableStateHandle<T> deserializeObject(String content) throws IOException {\n\t\tcheckNotNull(content, \"Content should not be null.\");\n\n\t\tfinal byte[] data = Base64.getDecoder().decode(content);\n\n\t\ttry {\n\t\t\treturn InstantiationUtil.deserializeObject(data, Thread.currentThread().getContextClassLoader());\n\t\t} catch (IOException | ClassNotFoundException e) {\n\t\t\tthrow new IOException(\"Failed to deserialize state handle from ConfigMap data \" +\n\t\t\t\tcontent + '.', e);\n\t\t}\n\t}\n\n\tprivate KubernetesException getConfigMapNotExistException() {\n\t\treturn new KubernetesException(\"ConfigMap \" + configMapName + \" does not exists. \" +\n\t\t\t\"It may be deleted externally.\");\n\t}\n\n\tprivate NotExistException getKeyNotExistException(String key) {\n\t\treturn new NotExistException(\"Could not find \" + key + \" in ConfigMap \" + configMapName);\n\t}\n\n\tprivate AlreadyExistException getKeyAlreadyExistException(String key) {\n\t\treturn new AlreadyExistException(key + \" already exists in ConfigMap \" + configMapName);\n\t}\n\n\tprivate String encodeStateHandle(byte[] serializedStoreHandle) {","sourceCodeStart":407,"sourceCodeEnd":443,"githubUrl":"https://github.com/zhisheng17/flink-learning/blob/d731cee7618021be56d132cc925102ffff8d75e6/flink-learning-k8s/flink-k8s/src/main/java/org/apache/flink/kubernetes/highavailability/KubernetesStateHandleStore.java#L407-L443","documentation":"deserializeObject() wraps IOException/ClassNotFoundException from InstantiationUtil.deserializeObject of the Base64-decoded ConfigMap value into this IOException. The stored state handle bytes cannot be turned back into an object, usually due to an incompatible classpath.","triggerScenarios":"getAndLock, getAllAndLock, releaseAndTryRemove, or releaseAndTryRemoveAll reading a ConfigMap entry whose payload was written by a different Flink/job version whose classes are missing or moved (ClassNotFoundException), or the Base64 payload is corrupted/truncated.","commonSituations":"Upgrading Flink or the job jar without draining old HA state; user-code classes present at write time but absent on the recovery classpath (no userCodeLoader); manual edits to ConfigMap data breaking Base64.","solutions":["Align the reader's classpath/Flink version with the writer that stored the state handle.","Ensure user-code classes are on the classpath (or in the job jar) of the recovering process — check Thread.currentThread().getContextClassLoader().","If the data is stale/incompatible, delete the offending ConfigMap key and let the job re-register its state.","Verify the ConfigMap value was not hand-edited; re-write it via the API instead."],"exampleFix":"// before: recovery fails with ClassNotFoundException for user classes\nkubectl get cm flink-ha -o yaml  # job-xxx payload from old jar\n// after: remove stale handle so the job re-registers\nkubectl delete key: kubectl patch cm flink-ha --type json -p '[{\"op\":\"remove\",\"path\":\"/data/job-xxx\"}]'","handlingStrategy":"validation","validationCode":"// verify the stored payload decodes and its classes resolve before recovery use\nbyte[] data = Base64.getDecoder().decode(content);\nClass.forName(stateHandleClassName, false, Thread.currentThread().getContextClassLoader());","typeGuard":null,"tryCatchPattern":"try {\n    StoredStateHandle h = store.getAndLock(key, lockIdentity);\n} catch (IOException e) {\n    if (ExceptionUtils.findThrowable(e, ClassNotFoundException.class).isPresent()) {\n        // stale/incompatible handle: purge key and re-register\n    }\n}","preventionTips":["Keep Flink and job-jar versions consistent between writer and recovery reader.","Bundle user-code classes in the recovering process classpath.","Do not hand-edit ConfigMap Base64 values.","Purge HA state after major upgrades (ha storage cleanup)."],"tags":["kubernetes","deserialization","state-handle","classpath"],"backgroundTag":"json-unmarshal-failed","analyzedSha":"d731cee7618021be56d132cc925102ffff8d75e6","analyzedAt":"2026-09-06T05:35:08.496Z","contentChangedAt":"2026-09-06T05:35:08.496Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}