{"record":{"id":"f471cf3eb726ef3f","repo":"apache/hadoop","slug":"can-t-load-state-from-image-in-a-running-secretman","errorCode":null,"errorMessage":"Can't load state from image in a running SecretManager.","messagePattern":"Can't load state from image in a running SecretManager\\.","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/security/token/delegation/DelegationTokenSecretManager.java","lineNumber":174,"sourceCode":"    DelegationTokenInformation info = currentTokens.get(dtId);\n    if (info != null) {\n      return info.getRenewDate();\n    } else {\n      throw new IOException(\"No delegation token found for this identifier\");\n    }\n  }\n\n  /**\n   * Load SecretManager state from fsimage.\n   * \n   * @param in input stream to read fsimage\n   * @throws IOException\n   */\n  public synchronized void loadSecretManagerStateCompat(DataInput in)\n      throws IOException {\n    if (running) {\n      // a safety check\n      throw new IOException(\n          \"Can't load state from image in a running SecretManager.\");\n    }\n    serializerCompat.load(in);\n  }\n\n  public static class SecretManagerState {\n    public final SecretManagerSection section;\n    public final List<SecretManagerSection.DelegationKey> keys;\n    public final List<SecretManagerSection.PersistToken> tokens;\n\n    public SecretManagerState(\n        SecretManagerSection s,\n        List<SecretManagerSection.DelegationKey> keys,\n        List<SecretManagerSection.PersistToken> tokens) {\n      this.section = s;\n      this.keys = keys;\n      this.tokens = tokens;\n    }","sourceCodeStart":156,"sourceCodeEnd":192,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/security/token/delegation/DelegationTokenSecretManager.java#L156-L192","documentation":"DelegationTokenSecretManager.loadSecretManagerStateCompat throws IOException when asked to deserialize token/key state from fsimage while the manager is already running (running == true). Loading persisted state initializes currentTokens/allKeys and must happen before startThreads() turns the manager on; afterwards, mutating from an image would clobber live state. This is a lifecycle guard against double-load or loading into a live secret manager.","triggerScenarios":"loadSecretManagerStateCompat(in) is invoked after start() — e.g., a NameNode (or a tool embedding the NN image loader) attempts to load a second fsimage, or code loads checkpoint state after the token manager already started its renewal/expiry threads.","commonSituations":"Custom tooling/tests that call FSImageFormat loadDelegateSections twice; NN rolling upgrade or checkpoint reload path that reuses a started DelegationTokenSecretManager; embedding HDFS image loading in an application that also runs a secret manager; framework code that loads images on a schedule.","solutions":["Load all fsimage state before starting the secret manager: call loadSecretManagerState* during image load, then startThreads()/start() exactly once.","If a second image must be processed, construct a new DelegationTokenSecretManager instance for the load (stop the old one first via stopThreads()).","In tests, reset the manager (create a fresh instance) between image loads instead of reusing one.","Audit custom FSImage consumers for accidental double invocation of the load path after startup."],"exampleFix":"// before\ndtSecretManager.startThreads();\n... \ndtSecretManager.loadSecretManagerStateCompat(in); // IOException: running SecretManager\n\n// after\n// load first, start once\ndtSecretManager.loadSecretManagerStateCompat(in);\ndtSecretManager.startThreads();","handlingStrategy":"validation","validationCode":"// Guard the load path: only load when not running\nif (dtSecretManager.isRunning()) {\n  throw new IllegalStateException(\"Refusing image load: use a fresh secret manager\");\n}\ndtSecretManager.loadSecretManagerStateCompat(in);","typeGuard":null,"tryCatchPattern":"try {\n  dtSecretManager.loadSecretManagerStateCompat(in);\n} catch (IOException e) {\n  if (e.getMessage().contains(\"running SecretManager\")) {\n    // lifecycle bug: create a new manager instance for this load\n    dtSecretManager = new DelegationTokenSecretManager(...);\n    dtSecretManager.loadSecretManagerStateCompat(in);\n  } else { throw e; }\n}","preventionTips":["Load all fsimage state strictly before startThreads()/start().","Create a fresh DelegationTokenSecretManager per image-load in tools and tests.","Never reuse a started manager for checkpoint loading."],"tags":["hdfs","delegation-token","security","lifecycle","fsimage"],"backgroundTag":"invalid-state-transition","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}