apache/hadoop · error · IOException

Failed to parse generation stamp: blockFile={blockFile}, met

Error message

Failed to parse generation stamp: blockFile={blockFile}, metaFile={metaFile}

What it means

FsDatasetUtil.parseGenerationStamp extracts the numeric generation stamp from a meta file name '<blockName>_<gs>.meta'. NumberFormatException wrapped in this IOException means the middle segment is not a long - a file name violating HDFS naming, produced by corruption, foreign tools, or hand-created files.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetUtil.java:194

      String metaFile = listdir[index + 1].getName();
      if (metaFile.startsWith(blockName)) {
        return Block.getGenerationStamp(metaFile);
      }
    }
    FsDatasetImpl.LOG.warn("Block " + blockFile + " does not have a metafile!");
    return HdfsConstants.GRANDFATHER_GENERATION_STAMP;
  }

  /** Find the corresponding meta data file from a given block file */
  static long parseGenerationStamp(File blockFile, File metaFile
      ) throws IOException {
    final String metaname = metaFile.getName();
    final String gs = metaname.substring(blockFile.getName().length() + 1,
        metaname.length() - Block.METADATA_EXTENSION.length());
    try {
      return Long.parseLong(gs);
    } catch(NumberFormatException nfe) {
      throw new IOException("Failed to parse generation stamp: blockFile="
          + blockFile + ", metaFile=" + metaFile, nfe);
    }
  }

  /**
   * Compute the checksum for a block file that does not already have
   * its checksum computed, and save it to dstMeta file.
   */
  public static void computeChecksum(File srcMeta, File dstMeta,
      File blockFile, int smallBufferSize, Configuration conf)
          throws IOException {
    Preconditions.checkNotNull(srcMeta);
    Preconditions.checkNotNull(dstMeta);
    Preconditions.checkNotNull(blockFile);
    // Create a dummy ReplicaInfo object pointing to the blockFile.
    ReplicaInfo wrapper = new FinalizedReplica(0, 0, 0, null, null) {
      @Override
      public URI getMetadataURI() {

View on GitHub (pinned to 2add963021)

Solutions

1. Rename the offending meta to the correct '<blockName>_<gs>.meta' shape if the true generation stamp is known (it must match the committed stamp).
2. Otherwise delete the malformed pair (block plus meta) and let re-replication restore the block, then fsck.
3. Find what writes foreign files into replica dirs and exclude those tools.
4. Run the DirectoryScanner to reconcile the volume map after cleanup.

if (!metaFile.getName().matches("^.+_\d+\.meta$")) {
  // malformed meta name: quarantine the file instead of parsing
  return;
}
long gs = FsDatasetUtil.parseGenerationStamp(blockFile, metaFile);

static boolean hasValidMetaName(File f) {
  return f != null && f.getName().matches("^.+_\d+\.meta$");
}

try {
  long gs = FsDatasetUtil.parseGenerationStamp(blockFile, metaFile);
} catch (IOException e) {
  // malformed name: quarantine the block/meta pair
}

When it happens

Trigger: Manually copied or renamed meta files; mangled filenames on a failing filesystem; legacy names missing the stamp; test fixtures with wrong names.

Common situations: Admins moving block files; disk corruption; exotic filesystems (Fuse, NFS) altering names; very old clusters upgraded in place.

Understand the failure class

• Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

Related errors

• Found more than one meta files: {matches}
• Replica gen stamp < block genstamp, block={block}, replica={
• Meta-data not found for {block}
• The meta file length {metaIn.getLength()} is less than the e
• ProvidedReplica does not support deleting metadata

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/ebf194121919fb1e. Report an issue: GitHub.