apache/hadoop · error · IOException

Cannot get FileChannel from Block InputStream meta file.

Error message

Cannot get FileChannel from Block InputStream meta file.

What it means

Thrown by NativePmemMappableBlockLoader.verifyChecksumAndMapBlock() (NativePmemMappableBlockLoader.java:132) when metaIn.getChannel() returns null while verifying checksums before committing a block into pmem cache. Like the sibling guards in MappableBlockLoader and the other loaders, FileInputStream.getChannel() never actually returns null on a stock JDK, making this a defensive, effectively unreachable branch.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/NativePmemMappableBlockLoader.java:132

    return mappableBlock;
  }

  /**
   * Verifies the block's checksum meanwhile map block to persistent memory.
   * This is an I/O intensive operation.
   */
  private void verifyChecksumAndMapBlock(POSIX.PmemMappedRegion region,
      long length, FileInputStream metaIn, FileChannel blockChannel,
      String blockFileName) throws IOException {
    // Verify the checksum from the block's meta file
    // Get the DataChecksum from the meta file header
    BlockMetadataHeader header =
        BlockMetadataHeader.readHeader(new DataInputStream(
            new BufferedInputStream(metaIn, BlockMetadataHeader
                .getHeaderSize())));
    try (FileChannel metaChannel = metaIn.getChannel()) {
      if (metaChannel == null) {
        throw new IOException("Cannot get FileChannel" +
            " from Block InputStream meta file.");
      }
      DataChecksum checksum = header.getChecksum();
      final int bytesPerChecksum = checksum.getBytesPerChecksum();
      final int checksumSize = checksum.getChecksumSize();
      final int numChunks = (8 * 1024 * 1024) / bytesPerChecksum;
      ByteBuffer blockBuf = ByteBuffer.allocate(numChunks * bytesPerChecksum);
      ByteBuffer checksumBuf = ByteBuffer.allocate(numChunks * checksumSize);
      // Verify the checksum
      int bytesVerified = 0;
      long mappedAddress = -1L;
      if (region != null) {
        mappedAddress = region.getAddress();
      }
      while (bytesVerified < length) {
        Preconditions.checkState(bytesVerified % bytesPerChecksum == 0,
            "Unexpected partial chunk before EOF.");
        assert bytesVerified % bytesPerChecksum == 0;

View on GitHub (pinned to 2add963021)

Solutions

Retest on a stock supported JDK without agents/instrumentation
If reproducible, add the stream class name to the exception via a debug patch and open a Hadoop JIRA
No data is corrupted - the load aborts before the mapping is published, and the cache directive retries later

Defensive patterns

Strategy: try-catch

Try / catch

try {
  verifyChecksumAndMapBlock(region, length, metaIn, blockChannel, blockFileName);
} catch (IOException e) {
  // loader's finally releases the mapped region; log and let the cache directive retry
  LOG.warn("pmem cache verify failed for {}: {}", blockFileName, e);
}

Prevention

Stock JDK only - the null-channel branch is unreachable for FileInputStream.getChannel()
Guard the real risks instead: valid meta files and healthy block replicas on the volume

When it happens

Trigger: Only a non-standard JVM/instrumented stream could trip it; the native pmem cache path itself passes a plain FileInputStream opened on the block meta file.

Common situations: Does not occur in practice; if seen, investigate the JVM rather than HDFS or pmem configuration.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/5acb00f56721f652. Report an issue: GitHub.