{"record":{"id":"03bd7f4c49f1acc5","repo":"apache/hadoop","slug":"invalid-number-of-blocks-the-maximum-number","errorCode":null,"errorMessage":"invalid number of blocks: {}.  The maximum number of blocks per file is {}","messagePattern":"invalid number of blocks: (.+?)\\.  The maximum number of blocks per file is (.+?)","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogOp.java","lineNumber":696,"sourceCode":"        }\n        // read clientId and callId\n        readRpcIds(in, logVersion);\n      } else {\n        this.clientName = \"\";\n        this.clientMachine = \"\";\n      }\n    }\n\n    static final public int MAX_BLOCKS = 1024 * 1024 * 64;\n    \n    private static Block[] readBlocks(\n        DataInputStream in,\n        int logVersion) throws IOException {\n      int numBlocks = in.readInt();\n      if (numBlocks < 0) {\n        throw new IOException(\"invalid negative number of blocks\");\n      } else if (numBlocks > MAX_BLOCKS) {\n        throw new IOException(\"invalid number of blocks: \" + numBlocks +\n            \".  The maximum number of blocks per file is \" + MAX_BLOCKS);\n      }\n      Block[] blocks = new Block[numBlocks];\n      for (int i = 0; i < numBlocks; i++) {\n        Block blk = new Block();\n        blk.readFields(in);\n        blocks[i] = blk;\n      }\n      return blocks;\n    }\n\n    public String stringifyMembers() {\n      StringBuilder builder = new StringBuilder();\n      builder.append(\"[length=\")\n          .append(length)\n          .append(\", inodeId=\")\n          .append(inodeId)\n          .append(\", path=\")","sourceCodeStart":678,"sourceCodeEnd":714,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogOp.java#L678-L714","documentation":"readBlocks() deserializes the block list of a block-carrying edit record (e.g. OP_ADD): it reads a 4-byte block count, then that many Block structs. The count must be between 0 and MAX_BLOCKS = 67,108,864 (1024*1024*64); anything larger is rejected so the NameNode never tries to allocate a multi-gigabyte array from a garbage count. A value above the cap means the stream is corrupt or misaligned, not that a real file had too many blocks.","triggerScenarios":"NameNode startup replay (or offlineEditsViewer / JournalNode tailing / bootstrapStandby) parses an edits segment where an earlier record was mis-parsed so the stream now sits on a body byte, or the count int itself is corrupt (torn write, bit rot). Also triggered when a segment written by an incompatible layout version is parsed with the wrong version header.","commonSituations":"NameNode restart after a crash or full journal disk left a partially written record; failing name.dir or journal disk; a stray non-edits file picked up from the current directory; rare downgrade/mixed-version storage dirs.","solutions":["Run 'hdfs namenode -recover' and answer yes to skip broken edits, then verify the namespace with 'hdfs fsck /' and immediately write a fresh checkpoint","Locate the corrupt transaction first: 'hdfs offlineEditsViewer -i <edits segment> -o /tmp/edits.xml' (it fails exactly at the bad record and prints its txid)","Replace the corrupt segment with a healthy copy from the QJM majority / SecondaryNameNode, or fall back to the last good fsimage plus valid edits","Test the journal and name disks (smartctl, fsck) and memtest the node before trusting them again"],"exampleFix":"# before\nhdfs namenode   # IOException: invalid number of blocks: 1893234546. The maximum number of blocks per file is 67108864\n# after\nhdfs namenode -recover   # skip the damaged edits, then: hdfs fsck / && hdfs dfsadmin -saveNamespace","handlingStrategy":"try-catch","validationCode":"# validate every segment before NameNode start / before replay\nfor f in /dfs/name/current/edits_*; do\n  hdfs offlineEditsViewer -i \"$f\" -o /dev/null || echo \"CORRUPT: $f\"\ndone","typeGuard":null,"tryCatchPattern":"try {\n  editLogInputStream.readOp(); // or fsImage.loadEdits(...) at startup\n} catch (IOException ioe) {\n  LOG.error(\"Corrupt edit record in \" + segment, ioe);\n  // re-run NameNode with -recover (skipBrokenEdits=true) to skip past it\n}","preventionTips":["Run a JournalNode quorum (3+) so a corrupt local copy is always outvoted by healthy ones","Keep name.dir and journal dirs on reliable, monitored storage; act on smartctl errors immediately","Take regular fsimage checkpoints so replay never touches long, old segments","After any unclean shutdown, run offlineEditsViewer over edits_inprogress before restarting the NameNode"],"tags":["hdfs","edit-log","namenode","corruption","blocks"],"backgroundTag":"edit-log-corruption","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}