apache/hadoop · error · FileNotFoundException

Meta file for {} not found.

Error message

Meta file for {} not found.

What it means

FileNotFoundException from DatanodeUtil.getMetaDataInputStream: a checksum read was requested for an ExtendedBlock, and FsDataset.getMetaDataInputStream(b) returned null, meaning the block's metadata (.meta) file does not exist on the DataNode. Block data files and their metadata files are pairs; a missing meta file makes checksum verification impossible, so read-time checksumming and block scanning fail for that replica.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DatanodeUtil.java:167

        res.add(DataStorage.BLOCK_SUBDIR_PREFIX + d1 + SEP +
            DataStorage.BLOCK_SUBDIR_PREFIX + d2);
      }
    }
    return res;
  }

  /**
   * @return the FileInputStream for the meta data of the given block.
   * @throws FileNotFoundException
   *           if the file not found.
   * @throws ClassCastException
   *           if the underlying input stream is not a FileInputStream.
   */
  public static FileInputStream getMetaDataInputStream(
      ExtendedBlock b, FsDatasetSpi<?> data) throws IOException {
    final LengthInputStream lin = data.getMetaDataInputStream(b);
    if (lin == null) {
      throw new FileNotFoundException("Meta file for " + b + " not found.");
    }
    return (FileInputStream)lin.getWrappedStream();
  }
}

View on GitHub (pinned to 2add963021)

Solutions

1. Let HDFS self-heal: mark the replica corrupt (hdfs fsck shows it; 'hdfs debug recoverLease' not needed - the scanner/client reports corruption and the NN re-replicates from healthy replicas)
2. Verify with 'hdfs fsck / -files -blocks -locations' that other replicas exist; if yes, the DN will invalidate the bad replica after the corruption report
3. On the DataNode, confirm the mismatch: ls the block dir for blk_<id>* and note the missing .meta sibling
4. If this is the last replica, escalate immediately: copy the block data file aside and seek recovery support before deleting anything

Example fix

# before: read fails - 'Meta file for blk_1073741830_1040 not found.'
ls /data/dfs/current/BP-*/subdir*/blk_1073741830*   # data file present, .meta missing

# after: confirm other replicas exist, report corrupt, let NN re-replicate
hdfs fsck / -files -blocks -locations | grep -A2 blk_1073741830
hdfs fsck / -move   # or let client corruption reports invalidate the replica
# DN deletes the bad replica; replication restores redundancy

// Before a checksummed read, verify the meta file exists next to the block file
File blockFile = new File(blockDir, b.getBlockName());
File metaFile = new File(blockDir,
    DatanodeUtil.getMetaName(b.getBlockName(), b.getGenerationStamp()));
if (!blockFile.exists() || !metaFile.exists()) {
  // replica is incomplete: skip it, report corruption, read another replica
  reportCorrupt(b);
}

try {
  FileInputStream meta = DatanodeUtil.getMetaDataInputStream(b, data);
} catch (FileNotFoundException e) {
  // meta missing for this replica: do not fail the read - fall back to another
  // replica via the DFSClient and let the corruption report trigger re-replication
  readFromOtherReplica(b);
}

When it happens

Trigger: Client read with checksums enabled, DataNode block scanner, or block verification (dfsclient reads calling getMetaDataInputStream) hits a replica whose blk_<id> exists but blk_<id>_<genstamp>.meta is absent: deleted by accident, lost in an unclean crash, truncated volume, or a pre-finalized rbw file whose meta was never created.

Common situations: Manual cleanup scripts deleting .meta files; disk corruption or partial restores; replicas written by very old Hadoop versions without meta regeneration on upgrade; interleaving of volume failures with re-replication.

Related errors

• Meta-data not found for {block}
• Checksum failed at {failedPos} for replica: {replica}
• Failed to move meta file for {b} from {metadataURI} to {dstm
• Failed to copy {srcReplica} metadata to {dstMeta}
• Failed to hardLink {srcReplica} metadata to {dstMeta}

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/b31230b6797ca69b. Report an issue: GitHub.