{"record":{"id":"8f763fff88e36785","repo":"apache/hadoop","slug":"invalid-metadata-header-version-version-can-on","errorCode":null,"errorMessage":"invalid metadata header version {version}.  Can only handle version 1.","messagePattern":"invalid metadata header version (.+?)\\.  Can only handle version 1\\.","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/shortcircuit/ShortCircuitReplica.java","lineNumber":131,"sourceCode":"\n  /**\n   * The monotonic time in nanoseconds at which the replica became evictable, or\n   * null if it is not evictable.\n   *\n   * Protected by the cache lock.\n   */\n  private Long evictableTimeNs = null;\n\n  public ShortCircuitReplica(ExtendedBlockId key,\n      FileInputStream dataStream, FileInputStream metaStream,\n      ShortCircuitCache cache, long creationTimeMs, Slot slot) throws IOException {\n    this.key = key;\n    this.dataStream = dataStream;\n    this.metaStream = metaStream;\n    this.metaHeader =\n          BlockMetadataHeader.preadHeader(metaStream.getChannel());\n    if (metaHeader.getVersion() != 1) {\n      throw new IOException(\"invalid metadata header version \" +\n          metaHeader.getVersion() + \".  Can only handle version 1.\");\n    }\n    this.cache = cache;\n    this.creationTimeMs = creationTimeMs;\n    this.slot = slot;\n  }\n\n  /**\n   * Decrement the reference count.\n   */\n  public void unref() {\n    cache.unref(this);\n  }\n\n  /**\n   * Check if the replica is stale.\n   *\n   * Must be called with the cache lock held.","sourceCodeStart":113,"sourceCodeEnd":149,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/shortcircuit/ShortCircuitReplica.java#L113-L149","documentation":"Thrown while constructing a ShortCircuitReplica during an HDFS short-circuit read. With short-circuit reads enabled, the datanode hands the client the replica's block and meta file descriptors, and the client preads the meta file header via BlockMetadataHeader.preadHeader(). This client only understands metadata header version 1 (checksum type plus bytes-per-checksum), so any other version is rejected with an IOException. A non-1 version means the meta file was written in an incompatible format or is corrupt or truncated.","triggerScenarios":"dfs.client.read.shortcircuit=true (with a domain socket at dfs.domain.socket.path) and ShortCircuitCache creating a replica whose meta stream preads a header with getVersion() != 1: a truncated meta file, corrupted first bytes on a failing disk, or a meta file produced by a datanode with a different metadata layout.","commonSituations":"Client and datanode running different Hadoop releases with short-circuit reads enabled; a replica corrupted by disk trouble or a crash; a block/meta pair left inconsistent across an upgrade.","solutions":["Run `hdfs fsck <path> -files -blocks -locations` to confirm block/meta health and force re-replication of corrupt replicas","Temporarily set dfs.client.read.shortcircuit=false on the client to fall back to normal datanode-mediated reads and confirm the meta file is at fault","Align client and datanode Hadoop versions; the meta format is version-coupled between them","On the datanode, hexdump the replica's .meta file header (first short must be 1); delete the bad replica so HDFS re-replicates it"],"exampleFix":"// before (client config)\n<property><name>dfs.client.read.shortcircuit</name><value>true</value></property>\n\n// after - isolate the fault by disabling short-circuit reads\n<property><name>dfs.client.read.shortcircuit</name><value>false</value></property>","handlingStrategy":"try-catch","validationCode":"// Validate the meta header before accepting a short-circuit replica\ntry (FileInputStream meta = new FileInputStream(metaFile)) {\n  BlockMetadataHeader header = BlockMetadataHeader.preadHeader(meta.getChannel());\n  if (header.getVersion() != 1) {\n    // refuse short-circuit for this block; use the normal read path\n  }\n}","typeGuard":null,"tryCatchPattern":"try {\n  // open/read with dfs.client.read.shortcircuit=true\n} catch (IOException e) {\n  if (e.getMessage() != null && e.getMessage().contains(\"invalid metadata header version\")) {\n    conf.setBoolean(\"dfs.client.read.shortcircuit\", false); // reopen via normal path\n  } else { throw e; }\n}","preventionTips":["Run client and datanode on the same Hadoop release when short-circuit reads are enabled","Schedule periodic hdfs fsck so corrupt replicas are re-replicated before readers hit them","Smoke-test short-circuit reads after every cluster upgrade before rolling the setting out"],"tags":["hdfs","short-circuit-read","block-metadata","version-mismatch","disk-corruption"],"backgroundTag":"file-format-version-mismatch","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-23T01:17:44.959Z"}