{"record":{"id":"65e0472ff4da68a5","repo":"apache/hadoop","slug":"got-unexpected-exception-message","errorCode":null,"errorMessage":"got unexpected exception {message}","messagePattern":"got unexpected exception (.+?)","errorType":"exception","errorClass":"java.io.IOException","httpStatus":null,"severity":"error","filePath":"hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogOp.java","lineNumber":5084,"sourceCode":"        try {\n          return decodeOp();\n        } catch (IOException e) {\n          in.reset();\n          if (!skipBrokenEdits) {\n            throw e;\n          }\n        } catch (RuntimeException e) {\n          // FSEditLogOp#decodeOp is not supposed to throw RuntimeException.\n          // However, we handle it here for recovery mode, just to be more\n          // robust.\n          in.reset();\n          if (!skipBrokenEdits) {\n            throw e;\n          }\n        } catch (Throwable e) {\n          in.reset();\n          if (!skipBrokenEdits) {\n            throw new IOException(\"got unexpected exception \" +\n                e.getMessage(), e);\n          }\n        }\n        // Move ahead one byte and re-try the decode process.\n        if (in.skip(1) < 1) {\n          return null;\n        }\n      }\n    }\n\n    void verifyTerminator() throws IOException {\n      /** The end of the edit log should contain only 0x00 or 0xff bytes.\n       * If it contains other bytes, the log itself may be corrupt.\n       * It is important to check this; if we don't, a stray OP_INVALID byte \n       * could make us stop reading the edit log halfway through, and we'd never\n       * know that we had lost data.\n       */\n      limiter.clearLimit();","sourceCodeStart":5066,"sourceCodeEnd":5102,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogOp.java#L5066-L5102","documentation":"Reader.readOp(skipBrokenEdits=false) catches Throwable from decodeOp(): FSEditLogOp#decodeOp is contractually IOException-only, so any other failure (NegativeArraySizeException, OutOfMemoryError from garbage-driven allocation, protobuf RuntimeExceptions) is wrapped as 'got unexpected exception <msg>' with the original as cause. With skipBrokenEdits=true (recovery mode) the same failure just advances one byte and retries the decode.","triggerScenarios":"Any edit-log corruption that makes decoding fail with a non-IOException during normal NameNode startup replay, JournalNode tailing, or checkpointing on a SecondaryNameNode; occasionally a genuine decoder bug such as an op type unknown to the running binary.","commonSituations":"Same root causes as the specific parse errors (torn writes, bit rot, version skew); the nested cause in the stack trace names the actual failure and usually maps to one of the more specific edit-log errors.","solutions":["Read the wrapped cause in the stack trace first — it identifies the real failure and the failing op","Run 'hdfs namenode -recover' (the skipBrokenEdits path) to get past the record byte-by-byte","Locate the bad txid with 'hdfs offlineEditsViewer' and recover from the last checkpoint or healthy journal copies","If the same segment decodes fine on another node or release, suspect version skew — replay it with the matching release and checkpoint"],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":"hdfs offlineEditsViewer -i <edits segment> -o /dev/null   # surfaces the underlying decode failure pre-start","typeGuard":null,"tryCatchPattern":"try {\n  reader.readOp(false); // skipBrokenEdits=false\n} catch (IOException e) {\n  Throwable cause = e.getCause(); // the 'unexpected exception' is always wrapped\n  LOG.error(\"decode failed on \" + segment, cause);\n  // classify via cause, then re-run with skipBrokenEdits=true (namenode -recover)\n}","preventionTips":["Always inspect the wrapped cause; it identifies the real record-level defect","Keep reader/decoder and writer on matching Hadoop releases to avoid unknown-op decode crashes","Validate segments out-of-band (offlineEditsViewer) after crashes and before upgrades"],"tags":["hdfs","edit-log","corruption","recovery","unexpected-exception"],"backgroundTag":"edit-log-corruption","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}