apache/hadoop · error · IOException

Could not start KeyId Counter

Error message

Could not start KeyId Counter

What it means

Error "Could not start KeyId Counter" thrown in apache/hadoop.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/token/delegation/ZKDelegationTokenSecretManager.java:282

      if (delTokSeqCounter != null) {
        delTokSeqCounter.start();
      }
      // the first batch range should be allocated during this starting window
      // by calling the incrSharedCount
      currentSeqNum = incrSharedCount(delTokSeqCounter, seqNumBatchSize);
      currentMaxSeqNum = currentSeqNum + seqNumBatchSize;
      LOG.info("Fetched initial range of seq num, from {} to {} ",
          currentSeqNum+1, currentMaxSeqNum);
    } catch (Exception e) {
      throw new IOException("Could not start Sequence Counter", e);
    }
    try {
      keyIdSeqCounter = new SharedCount(zkClient, ZK_DTSM_KEYID_ROOT, 0);
      if (keyIdSeqCounter != null) {
        keyIdSeqCounter.start();
      }
    } catch (Exception e) {
      throw new IOException("Could not start KeyId Counter", e);
    }
    try {
      createPersistentNode(ZK_DTSM_MASTER_KEY_ROOT);
      createPersistentNode(ZK_DTSM_TOKENS_ROOT);
    } catch (Exception e) {
      throw new RuntimeException("Could not create ZK paths");
    }
    try {
      keyCache = CuratorCache.bridgeBuilder(zkClient, ZK_DTSM_MASTER_KEY_ROOT)
          .build();
      CuratorCacheListener keyCacheListener = CuratorCacheListener.builder()
          .forCreatesAndChanges((oldNode, node) -> {
            try {
              processKeyAddOrUpdate(node.getData());
            } catch (IOException e) {
              LOG.error("Error while processing Curator keyCacheListener "
                  + "NODE_CREATED / NODE_CHANGED event");
              throw new UncheckedIOException(e);

View on GitHub (pinned to 2add963021)

Solutions

1. Starting the ZK shared key-id counter failed. Check ZK connectivity and ACL permissions on the key-id counter znode.

When it happens

Trigger: Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/token/delegation/ZKDelegationTokenSecretManager.java:282 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/731510840a6d6e55. Report an issue: GitHub.