{"record":{"id":"a967cc5ee056587b","repo":"apache/hadoop","slug":"can-t-update-persisted-delegation-token-renewal-to","errorCode":null,"errorMessage":"Can't update persisted delegation token renewal to a running SecretManager.","messagePattern":"Can't update persisted delegation token renewal to a running SecretManager\\.","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/security/token/delegation/DelegationTokenSecretManager.java","lineNumber":326,"sourceCode":"   * @throws IOException\n   */\n  public synchronized void updatePersistedMasterKey(DelegationKey key)\n      throws IOException {\n    addKey(key);\n  }\n  \n  /**\n   * Update the token cache with renewal record in edit logs.\n   * \n   * @param identifier DelegationTokenIdentifier of the renewed token\n   * @param expiryTime expirty time in milliseconds\n   * @throws IOException\n   */\n  public synchronized void updatePersistedTokenRenewal(\n      DelegationTokenIdentifier identifier, long expiryTime) throws IOException {\n    if (running) {\n      // a safety check\n      throw new IOException(\n          \"Can't update persisted delegation token renewal to a running SecretManager.\");\n    }\n    DelegationTokenInformation info = null;\n    info = currentTokens.get(identifier);\n    if (info != null) {\n      int keyId = identifier.getMasterKeyId();\n      byte[] password = createPassword(identifier.getBytes(), allKeys\n          .get(keyId).getKey());\n      currentTokens.put(identifier, new DelegationTokenInformation(expiryTime,\n          password, getTrackingIdIfEnabled(identifier)));\n    }\n  }\n\n  /**\n   *  Update the token cache with the cancel record in edit logs\n   *  \n   *  @param identifier DelegationTokenIdentifier of the canceled token\n   *  @throws IOException","sourceCodeStart":308,"sourceCodeEnd":344,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/security/token/delegation/DelegationTokenSecretManager.java#L308-L344","documentation":"DelegationTokenSecretManager.updatePersistedTokenRenewal throws IOException when an OP_DELEGATION_TOKEN_RENEW record from edit logs is applied while the secret manager is already running. Renewal replay is part of image/log loading; a running manager must only see renewals through the live renewToken RPC path. The check (running == true) catches startup ordering bugs where replay happens after the manager went live.","triggerScenarios":"updatePersistedTokenRenewal(identifier, expiryTime) is invoked after startThreads(), i.e., edit-log replay (or a tool re-applying renew records) runs against a started secret manager.","commonSituations":"NameNode startup enabling the RPC server before replay completes; custom log-replay tooling pointed at a live manager; tests that start the manager then replay logs; double-started secret manager instances.","solutions":["Reorder startup: finish fsimage load and full edit-log replay before starting the secret manager threads / RPC server.","Use a separate, non-started DelegationTokenSecretManager for any offline replay of renew records.","In tests, create a fresh manager per replay pass; never reuse a started one.","Audit for double start of the manager (extra startThreads call) in custom bootstrap code."],"exampleFix":"// before\ndtSecretManager.startThreads();\nreplayEditLogOp(OP_DELEGATION_TOKEN_RENEW, id, expiry); // IOException\n\n// after\nreplayEditLogOp(OP_DELEGATION_TOKEN_RENEW, id, expiry); // manager not yet running\ndtSecretManager.startThreads();","handlingStrategy":"validation","validationCode":"assert !dtSecretManager.isRunning() : \"renewal replay must precede startThreads()\";\ndtSecretManager.updatePersistedTokenRenewal(identifier, expiryTime);","typeGuard":null,"tryCatchPattern":"try {\n  dtSecretManager.updatePersistedTokenRenewal(identifier, expiryTime);\n} catch (IOException e) {\n  if (e.getMessage().contains(\"running SecretManager\")) {\n    // reorder: replay edits first, then start manager; or replay into fresh instance\n  } else { throw e; }\n}","preventionTips":["Complete edit-log replay before starting the secret manager.","Use a non-running manager for offline replay of renew records.","Test startup ordering so replay always precedes activation."],"tags":["hdfs","delegation-token","edit-log","lifecycle","startup"],"backgroundTag":"invalid-state-transition","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}