{"record":{"id":"148e4a638d3c07cb","repo":"apache/hadoop","slug":"another-name-is-running","errorCode":null,"errorMessage":"Another {name} is running.","messagePattern":"Another (.+?) is running\\.","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/balancer/NameNodeConnector.java","lineNumber":214,"sourceCode":"    this.getBlocksToStandby = !conf.getBoolean(\n        DFSConfigKeys.DFS_NAMENODE_GETBLOCKS_CHECK_OPERATION_KEY,\n        DFSConfigKeys.DFS_NAMENODE_GETBLOCKS_CHECK_OPERATION_DEFAULT);\n    this.config = conf;\n\n    this.fs = (DistributedFileSystem)FileSystem.get(nameNodeUri, conf);\n\n    final NamespaceInfo namespaceinfo = namenode.versionRequest();\n    this.blockpoolID = namespaceinfo.getBlockPoolID();\n\n    final FsServerDefaults defaults = fs.getServerDefaults(new Path(\"/\"));\n    this.keyManager = new KeyManager(blockpoolID, namenode,\n        defaults.getEncryptDataTransfer(), conf);\n    // if it is for test, we do not create the id file\n    if (checkOtherInstanceRunning) {\n      out = checkAndMarkRunning();\n      if (out == null) {\n        // Exit if there is another one running.\n        throw new IOException(\"Another \" + name + \" is running.\");\n      }\n    }\n  }\n\n  public NameNodeConnector(String name, URI nameNodeUri, String nsId,\n                           Path idPath, List<Path> targetPaths,\n                           Configuration conf, int maxNotChangedIterations)\n      throws IOException {\n    this(name, nameNodeUri, idPath, targetPaths, conf, maxNotChangedIterations);\n    this.nsId = nsId;\n  }\n\n  public DistributedFileSystem getDistributedFileSystem() {\n    return fs;\n  }\n\n  /** @return the block pool ID */\n  public String getBlockpoolID() {","sourceCodeStart":196,"sourceCodeEnd":232,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/balancer/NameNodeConnector.java#L196-L232","documentation":"NameNodeConnector's constructor calls checkAndMarkRunning, which enforces a single balancer instance per namespace via a lock file (default /system/balancer.id in HDFS). If the file exists it first tries fs.append(idPath): append fails while another instance holds the file's HDFS lease, so a failed append means a live peer and the constructor throws IOException('Another Balancer is running.'). A stale file from a cleanly-exited or killed balancer is deleted and reused, so seeing this error means a lease is genuinely held (or recovering).","triggerScenarios":"Starting a second 'hdfs balancer' against the same namespace while the first still runs - overlapping cron jobs, two operators on different gateways, or a wrapper script that spawns the balancer twice.","commonSituations":"Cron schedules that overlap when runs are long; an old balancer process left running on another gateway; immediately re-running the balancer after killing one process (lease not yet recovered on the NameNode).","solutions":["Find and stop the other running instance (check 'jps' / process lists on all gateways for a Balancer process), then rerun","If runs are long, serialize them in your scheduler (lock/flock or a single cron owner) instead of relying on this error","If you are certain nothing runs and the lease is stale, wait for HDFS lease recovery, or delete /system/balancer.id with hdfs dfs -rm after double-checking no balancer is active"],"exampleFix":"# before: overlapping cron\n*/30 * * * * hdfs balancer\n\n# after: serialized run with a local lock\n30 1 * * * flock -n /tmp/balancer.lock hdfs balancer","handlingStrategy":"try-catch","validationCode":"// best-effort pre-check (racy; the append-probe is authoritative)\nFileSystem fs = FileSystem.get(conf);\nif (fs.exists(new Path(\"/system/balancer.id\"))) {\n  try { IOUtils.closeStream(fs.append(new Path(\"/system/balancer.id\"))); }\n  catch (IOException leaseHeld) { throw new IllegalStateException(\"A balancer appears to be running\"); }\n}","typeGuard":null,"tryCatchPattern":"try { nnc = new NameNodeConnector(name, uri, idPath, targetPaths, conf, maxIdle); }\ncatch (IOException e) {\n  if (e.getMessage().startsWith(\"Another\")) { LOG.warn(\"{} - skipping scheduled run\", e.getMessage()); return EXIT_ALREADY_RUNNING; }\n  throw e;\n}","preventionTips":["Serialize balancer runs with a scheduler lock (flock/single cron owner) instead of relying on concurrent-start failures","Treat 'Another ... is running' as a no-op exit code in automation, not an incident","Before force-cleaning /system/balancer.id, verify no Balancer JVM on any gateway"],"tags":["hdfs","balancer","singleton-lock","hdfs-lease","concurrency"],"backgroundTag":"duplicate-instance-lock","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}