apache/hadoop · error · IOException

Block InputStream meta file has no FileChannel.

Error message

Block InputStream meta file has no FileChannel.

What it means

Thrown by MappableBlockLoader.verifyChecksum() (MappableBlockLoader.java:140), the shared checksum-verification step for centralized cache loaders, when metaIn.getChannel() returns null for the block meta file's InputStream. It is a defensive guard: FileInputStream.getChannel() in the JDK always lazily creates and returns a channel, so with the FileInputStreams HDFS passes this branch is effectively unreachable in practice.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/MappableBlockLoader.java:140

   */
  void shutdown() {
    // Do nothing.
  }

  /**
   * Verifies the block's checksum. This is an I/O intensive operation.
   */
  protected void verifyChecksum(long length, FileInputStream metaIn,
      FileChannel blockChannel, String blockFileName) throws IOException {
    // Verify the checksum from the block's meta file
    // Get the DataChecksum from the meta file header
    BlockMetadataHeader header =
        BlockMetadataHeader.readHeader(new DataInputStream(
            new BufferedInputStream(metaIn, BlockMetadataHeader
                .getHeaderSize())));
    try (FileChannel metaChannel = metaIn.getChannel()) {
      if (metaChannel == null) {
        throw new IOException(
            "Block InputStream meta file has no FileChannel.");
      }
      DataChecksum checksum = header.getChecksum();
      final int bytesPerChecksum = checksum.getBytesPerChecksum();
      final int checksumSize = checksum.getChecksumSize();
      final int numChunks = (8 * 1024 * 1024) / bytesPerChecksum;
      ByteBuffer blockBuf = ByteBuffer.allocate(numChunks * bytesPerChecksum);
      ByteBuffer checksumBuf = ByteBuffer.allocate(numChunks * checksumSize);
      // Verify the checksum
      int bytesVerified = 0;
      while (bytesVerified < length) {
        Preconditions.checkState(bytesVerified % bytesPerChecksum == 0,
            "Unexpected partial chunk before EOF");
        assert bytesVerified % bytesPerChecksum == 0;
        int bytesRead = fillBuffer(blockChannel, blockBuf);
        if (bytesRead == -1) {
          throw new IOException("checksum verification failed: premature EOF");
        }

View on GitHub (pinned to 2add963021)

Solutions

1. Treat occurrence as a JVM-level anomaly: check for non-standard JDK builds or agent instrumentation and retest on a stock supported JDK
2. Update/patch the JDK; open a Hadoop JIRA if it reproduces on stock JDK with a normal dfs -files cache workload
3. The IOException propagates to FsDatasetCache, which uncaches the block - retry the cache directive after the JDK fix

// FsDatasetCache already wraps caching in a handler that uncaches on IOException;
// rely on that and let the directive retry after the environment is fixed
try {
  mappableBlock = loader.load(length, blockIn, metaIn, blockFileName, key);
} catch (IOException e) {
  LOG.warn("Caching failed for {}: {}", key, e);
  // uncache path frees the reservation
}

When it happens

Trigger: Only reachable if a channel genuinely cannot be obtained - in practice a JVM/JDK anomaly or a custom InputStream subclass passed instead of FileInputStream. Normal HDFS cache loads (MemoryMappableBlockLoader, PmemMappableBlockLoader) never produce it.

Common situations: Essentially never in production; if it appears, suspect a patched/buggy JDK or non-standard code injecting streams into the caching path.

Related errors

• Block InputStream has no FileChannel.
• Block InputStream has no FileChannel.
• Cannot get FileChannel from Block InputStream meta file.
• Block InputStream has no FileChannel.
• Cannot start datanode because the configured max locked memo

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/1d964aa0178fa513. Report an issue: GitHub.