apache/hadoop · error · IllegalArgumentException

Block pool {bpid} is not found

Error message

Block pool {bpid} is not found

What it means

BlockPoolTokenSecretManager.get(bpid) throws IllegalArgumentException when asked for the BlockTokenSecretManager of a block pool id that was never registered via addBlockPool. The manager is a bpid -> secret manager map keyed at DN/NN startup as block pools are initialized; querying an unknown bpid means the caller is handling blocks or tokens for a pool this process does not know about.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/security/token/block/BlockPoolTokenSecretManager.java:56

    SecretManager<BlockTokenIdentifier> {
  
  private final Map<String, BlockTokenSecretManager> map =
      new ConcurrentHashMap<>();

  /**
   * Add a block pool Id and corresponding {@link BlockTokenSecretManager} to map
   * @param bpid block pool Id
   * @param secretMgr {@link BlockTokenSecretManager}
   */
  public void addBlockPool(String bpid, BlockTokenSecretManager secretMgr) {
    map.put(bpid, secretMgr);
  }

  @VisibleForTesting
  public BlockTokenSecretManager get(String bpid) {
    BlockTokenSecretManager secretMgr = map.get(bpid);
    if (secretMgr == null) {
      throw new IllegalArgumentException(
          "Block pool " + bpid + " is not found");
    }
    return secretMgr;
  }
  
  public boolean isBlockPoolRegistered(String bpid) {
    return map.containsKey(bpid);
  }

  /** Return an empty BlockTokenIdentifer */
  @Override
  public BlockTokenIdentifier createIdentifier() {
    return new BlockTokenIdentifier();
  }

  @Override
  public byte[] createPassword(BlockTokenIdentifier identifier) {
    return get(identifier.getBlockPoolId()).createPassword(identifier);

View on GitHub (pinned to 2add963021)

Solutions

1. Register the pool first: call addBlockPool(bpid, secretManager) (in DN terms, ensure the BPOfferService for that pool completed initialization) before any get().
2. Verify the bpid string matches the actual pool — print FsDatasetTestUtil/cluster.getBlockPoolId() in tests instead of hardcoding.
3. Guard call sites with isBlockPoolRegistered(bpid) and skip/handle unknown pools gracefully.
4. If a stale storage dir for a removed namespace is involved, clear the stale BP directory under the DataNode storage so the unknown pool is not surfaced again.

Example fix

// before
BlockTokenSecretManager sm = bpTokenSecretManager.getBlockTokenSecretManager("BP-fake");
// IllegalArgumentException: Block pool BP-fake is not found

// after
if (bpTokenSecretManager.isBlockPoolRegistered(bpid)) {
  BlockTokenSecretManager sm = bpTokenSecretManager.getBlockTokenSecretManager(bpid);
} else {
  bpTokenSecretManager.addBlockPool(bpid, new BlockTokenSecretManager(...));
}

if (!bpTokenSecretManager.isBlockPoolRegistered(bpid)) {
  // unknown pool: register it or reject the request before calling get()
  throw new UnknownBlockPoolException(bpid);
}
BlockTokenSecretManager sm = bpTokenSecretManager.getBlockTokenSecretManager(bpid);

try {
  BlockTokenSecretManager sm = bpTokenSecretManager.getBlockTokenSecretManager(bpid);
} catch (IllegalArgumentException e) {
  if (e.getMessage().endsWith("is not found")) {
    // handle unknown block pool: log bpid, skip/reject block op
  } else { throw e; }
}

When it happens

Trigger: get(bpid) is called (e.g., during block token generation or validation in tests, or via BlockPoolTokenSecretManager public API) before addBlockPool for that bpid ran, or with a typo'd/wrong bpid. In production NN code the bpid comes from the namespace; in tests it is frequently a hardcoded string.

Common situations: Unit tests using a fake/typo'd block pool id (e.g., 'BP-123' vs the real generated 'BP-<random>-<ts>') without registering it; calling token APIs during DataNode init before registerBlockPool; reading a block from an old cluster's storage dir after re-registration; mixed-up test fixtures reusing a stale bpid.

Related errors

• Can't re-compute encryption key for nonce, since the require
• currentKey hasn't been initialized.
• Cannot get access token since BlockKeyUpdater is not running
• Got access token error, status message ${message}, ${logInfo
• Fetch of delegation token failed

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/b0a1a6bed9b1bd9e. Report an issue: GitHub.