apache/hadoop · error · IOException

Block cannot be null

Error message

Block cannot be null

What it means

Client-side translator for the InMemoryAliasMap protocol, used by HDFS Provided Storage to map a Block to its ProvidedStorageLocation (path, offset, length, version) on an external store via RPC. read() refuses a null Block because the protobuf ReadRequestProto requires a key. This is a caller-input precondition failure, not a server-side fault.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/InMemoryAliasMapProtocolClientSideTranslatorPB.java:165

        .collect(Collectors.toList());
    BlockProto nextMarker = response.getNextMarker();

    if (nextMarker.isInitialized()) {
      return new InMemoryAliasMap.IterationResult(fileRegions,
          Optional.of(PBHelperClient.convert(nextMarker)));
    } else {
      return new InMemoryAliasMap.IterationResult(fileRegions,
          Optional.empty());
    }
  }

  @Nonnull
  @Override
  public Optional<ProvidedStorageLocation> read(@Nonnull Block block)
      throws IOException {

    if (block == null) {
      throw new IOException("Block cannot be null");
    }
    ReadRequestProto request =
        ReadRequestProto
            .newBuilder()
            .setKey(PBHelperClient.convert(block))
            .build();
    ReadResponseProto response = ipc(() -> rpcProxy.read(null, request));

    ProvidedStorageLocationProto providedStorageLocation =
        response.getValue();
    if (providedStorageLocation.isInitialized()) {
      return Optional.of(PBHelperClient.convert(providedStorageLocation));
    }
    return Optional.empty();

  }

  @Override

View on GitHub (pinned to 2add963021)

Solutions

1. Fix the caller to construct and pass a non-null Block (built from the block id being resolved).
2. If absence is legitimate in your flow, check the alias map with contains()/Optional before calling read().
3. Add a unit test or static null-check (Nullaway/CheckerFramework) asserting the lookup path never passes null.

Example fix

// before
ProvidedStorageLocation loc = aliasMap.read(block).get(); // block may be null

// after
if (block == null) {
  throw new IllegalArgumentException("block required for alias-map lookup");
}
Optional<ProvidedStorageLocation> loc = aliasMap.read(block);

if (block == null) {
  throw new IllegalArgumentException(
      "block must be non-null before alias-map read()");
}
Optional<ProvidedStorageLocation> loc = aliasMap.read(block);

static boolean isResolvableBlock(Block b) {
  return b != null && b.getBlockId() > 0;
}

try {
  Optional<ProvidedStorageLocation> loc = aliasMap.read(block);
} catch (IOException e) {
  if (e.getMessage() != null && e.getMessage().contains("Block cannot be null")) {
    // caller bug: fix the producer of the null Block; do not retry
    throw new IllegalStateException("alias-map read called with null Block", e);
  }
  throw e;
}

When it happens

Trigger: Calling InMemoryAliasMap.read(block) with null: an alias-map lookup loop passing an uninitialized Block, a test invoking the translator directly, or custom ProvidedStorage code that drops the Block before resolution.

Common situations: Developing or integrating Provided Storage (S3, Azure, LocalProv) with a shared InMemoryAliasMap/LevelDBAliasMap server; almost always a caller bug rather than a configuration problem.

Related errors

• Provided block and location cannot be null
• InMemoryAliasMap location is null
• Unable to create missing aliasmap location: {levelDBpath}
• Failed to fully delete compressed aliasmap {compressedAliasM
• Unable to create aliasmap snapshot directory {newLevelDBDir}

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/bf47449e4047abe0. Report an issue: GitHub.