apache/hadoop · error · FileNotFoundException

Meta-data not found for {block}

Error message

Meta-data not found for {block}

What it means

BlockSender needs the block's meta file to verify or send checksums (verifyChecksum || sendChecksum and corruptChecksumOk false). When datanode.data.getMetaDataInputStream(block) returns null, there is no checksum file alongside the block file, so a FileNotFoundException is thrown and (per the catch block) the datanode invalidates the missing block with the NameNode so it can be re-replicated.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockSender.java:323

      /* 
       * (corruptChecksumOK, meta_file_exist): operation
       * True,   True: will verify checksum  
       * True,  False: No verify, e.g., need to read data from a corrupted file 
       * False,  True: will verify checksum
       * False, False: throws IOException file not found
       */
      DataChecksum csum = null;
      if (verifyChecksum || sendChecksum) {
        LengthInputStream metaIn = null;
        boolean keepMetaInOpen = false;
        try {
          DataNodeFaultInjector.get().throwTooManyOpenFiles();
          metaIn = datanode.data.getMetaDataInputStream(block);
          if (!corruptChecksumOk || metaIn != null) {
            if (metaIn == null) {
              //need checksum but meta-data not found
              throw new FileNotFoundException("Meta-data not found for " +
                  block);
            }

            // The meta file will contain only the header if the NULL checksum
            // type was used, or if the replica was written to transient storage.
            // Also, when only header portion of a data packet was transferred
            // and then pipeline breaks, the meta file can contain only the
            // header and 0 byte in the block data file.
            // Checksum verification is not performed for replicas on transient
            // storage.  The header is important for determining the checksum
            // type later when lazy persistence copies the block to non-transient
            // storage and computes the checksum.
            int expectedHeaderSize = BlockMetadataHeader.getHeaderSize();
            if (!replica.isOnTransientStorage() &&
                metaIn.getLength() >= expectedHeaderSize) {
              checksumIn = new DataInputStream(new BufferedInputStream(
                  metaIn, IO_FILE_BUFFER_SIZE));

View on GitHub (pinned to 2add963021)

Solutions

1. Check whether the .meta file exists in the block's finalizado directory on disk (blk_<id>_<gen>.meta next to blk_<id>).
2. If lost, the DN already reports the replica missing/invalid to the NN — let re-replication restore a complete replica from another datanode.
3. Audit for the root cause: failed disks, improper rsync of storage dirs, or operators deleting meta files.
4. Never reconstruct block dirs by hand without their .meta files; use re-replication or backups instead.

static boolean isMissingMeta(IOException e) {
  return e instanceof FileNotFoundException &&
      e.getMessage() != null && e.getMessage().startsWith("Meta-data not found for");
}

try {
  sender = new BlockSender(block, startOffset, length, /*corruptChecksumOk*/false, ...);
} catch (FileNotFoundException e) {
  if (isMissingMeta(e)) {
    reportReplicaCorrupt(block);      // DN invalidates it, NN re-replicates
    reader = readFromOtherReplica();  // fall back to a complete replica
  } else {
    throw e;
  }
}

When it happens

Trigger: The block file exists but its .meta companion was deleted or never written: interrupted finalize, meta file lost in a disk incident, an improperly copied/moved block directory, or a transient/lazy-persist replica that legitimately has no checksums being read with checksum verification required.

Common situations: Manual block-dir copying or disk repair that dropped .meta files; filesystem corruption hitting only small meta files; reading transient (RAM-disk lazy-persist) replicas through a path that demands checksums.

Related errors

• The meta file length {metaIn.getLength()} is less than the e
• Checksum failed at {failedPos} for replica: {replica}
• Meta file not found, blockFile={blockFile}
• Replica gen stamp < block genstamp, block={block}, replica={
• Replica is not readable, block={block}, replica={replica}

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/71cea84b2af2435d. Report an issue: GitHub.