{"record":{"id":"66018e93e92c4ea8","repo":"apache/hadoop","slug":"the-meta-file-length-metain-getlength-is-less","errorCode":null,"errorMessage":"The meta file length {metaIn.getLength()} is less than the expected length {expectedHeaderSize}","messagePattern":"The meta file length (.+?) is less than the expected length (.+?)","errorType":"exception","errorClass":"CorruptMetaHeaderException","httpStatus":null,"severity":"error","filePath":"hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockSender.java","lineNumber":349,"sourceCode":"            // header and 0 byte in the block data file.\n            // Checksum verification is not performed for replicas on transient\n            // storage.  The header is important for determining the checksum\n            // type later when lazy persistence copies the block to non-transient\n            // storage and computes the checksum.\n            int expectedHeaderSize = BlockMetadataHeader.getHeaderSize();\n            if (!replica.isOnTransientStorage() &&\n                metaIn.getLength() >= expectedHeaderSize) {\n              checksumIn = new DataInputStream(new BufferedInputStream(\n                  metaIn, IO_FILE_BUFFER_SIZE));\n\n              csum = BlockMetadataHeader.readDataChecksum(checksumIn, block);\n              keepMetaInOpen = true;\n            } else if (!replica.isOnTransientStorage() &&\n                metaIn.getLength() < expectedHeaderSize) {\n              LOG.warn(\"The meta file length {} is less than the expected \" +\n                  \"header length {}, indicating the meta file is corrupt\",\n                  metaIn.getLength(), expectedHeaderSize);\n              throw new CorruptMetaHeaderException(\"The meta file length \"+\n                  metaIn.getLength()+\" is less than the expected length \"+\n                  expectedHeaderSize);\n            }\n          } else {\n            LOG.warn(\"Could not find metadata file for \" + block);\n          }\n        } catch (FileNotFoundException e) {\n          if ((e.getMessage() != null) && !(e.getMessage()\n              .contains(\"Too many open files\"))) {\n            datanode.data.invalidateMissingBlock(block.getBlockPoolId(),\n                block.getLocalBlock());\n          }\n          throw e;\n        } finally {\n          if (!keepMetaInOpen) {\n            IOUtils.closeStream(metaIn);\n          }\n        }","sourceCodeStart":331,"sourceCodeEnd":367,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockSender.java#L331-L367","documentation":"Every block meta file starts with a fixed-size BlockMetadataHeader (version + checksum-type/chunk info, expectedHeaderSize bytes). If the existing meta file is shorter than that header, it is truncated garbage, so CorruptMetaHeaderException is thrown after a WARN that the meta file is corrupt — the checksum layout cannot even be parsed, let alone verified.","triggerScenarios":"metaIn.getLength() < expectedHeaderSize for a non-transient replica: the .meta file was truncated by a crash mid-write, disk corruption, or a partial copy of the storage directory; reading then demands checksum verification.","commonSituations":"Power loss truncating meta files; bad disk sectors hitting the (small) meta file; operators or backup tools that copied block files incompletely.","solutions":["Treat the replica as corrupt: remove the truncated blk_*.meta (and its block file) so the datanode reports the replica invalid and the NN re-replicas the block.","Run a filesystem check (xfs/zfs scrub, SMART) on the affected volume — one truncated meta often means a failing disk.","Confirm re-replication completes with 'hdfs fsck <file> -files -blocks -locations'.","Prevent: avoid hand-copies of storage dirs and ensure meta files are included/complete in any DR procedure."],"exampleFix":null,"handlingStrategy":"fallback","validationCode":null,"typeGuard":"static boolean isCorruptMetaHeader(IOException e) {\n  return e instanceof CorruptMetaHeaderException;\n}","tryCatchPattern":"try {\n  sender = new BlockSender(block, 0, -1, /*verifyChecksum*/true, ...);\n} catch (CorruptMetaHeaderException e) {\n  markReplicaCorruptAndRequeue(block); // remove replica, NN re-replicates\n  sender = buildFromHealthyReplica();\n}","preventionTips":["Use filesystem-level integrity (ZFS/xfs reflink backups, RAID scrub) so meta truncation is caught by the OS first.","Avoid partial rsync/copies of dfs.datanode.dir content between volumes or hosts.","Watch for the paired WARN 'indicating the meta file is corrupt' and treat it as a disk-health incident."],"tags":["hdfs","datanode","read-path","metadata","disk-corruption"],"backgroundTag":"corrupt-block-metadata","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-23T01:17:44.959Z"}