{"record":{"id":"36621728289f9829","repo":"apache/hadoop","slug":"federated-metrics-is-not-initialized","errorCode":null,"errorMessage":"Federated metrics is not initialized","messagePattern":"Federated metrics is not initialized","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/metrics/NamenodeBeanMetrics.java","lineNumber":178,"sourceCode":"    if (fsStateBeanName != null) {\n      MBeans.unregister(fsStateBeanName);\n      fsStateBeanName = null;\n    }\n    if (nnInfoBeanName != null) {\n      MBeans.unregister(nnInfoBeanName);\n      nnInfoBeanName = null;\n    }\n    // Remove the NameNode status bean\n    if (nnStatusBeanName != null) {\n      MBeans.unregister(nnStatusBeanName);\n      nnStatusBeanName = null;\n    }\n  }\n\n  private RBFMetrics getRBFMetrics() throws IOException {\n    RBFMetrics metrics = getRouter().getMetrics();\n    if (metrics == null) {\n      throw new IOException(\"Federated metrics is not initialized\");\n    }\n    return metrics;\n  }\n\n  /////////////////////////////////////////////////////////\n  // NameNodeMXBean\n  /////////////////////////////////////////////////////////\n\n  @Override\n  public String getVersion() {\n    return VersionInfo.getVersion() + \", r\" + VersionInfo.getRevision();\n  }\n\n  @Override\n  public String getSoftwareVersion() {\n    return VersionInfo.getVersion();\n  }\n","sourceCodeStart":160,"sourceCodeEnd":196,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/metrics/NamenodeBeanMetrics.java#L160-L196","documentation":"Thrown by NamenodeBeanMetrics.getRBFMetrics() when the Router's RBFMetrics object is null, meaning the Router has not registered its federation metrics (JMX) yet. The Router only creates RBFMetrics when its metrics service (RouterMetricsService) initializes, so this indicates the Router is not fully started, is stopping, or was constructed without the metrics service. It is an IOException surfacing through the NameNodeMXBean facade that proxies federated metrics to JMX clients.","triggerScenarios":"Calling any NameNodeMXBean method on the Router (e.g. JMX query on NamenodeBeanMetrics, getRouter()/getMembers()/getNameservices() MBean attributes) before Router.serviceStart() completes or after serviceStop(); unit tests instantiating NamenodeBeanMetrics with a Router stub whose getMetrics() returns null; a Router whose state store connection failed during init so metrics were never bound.","commonSituations":"Monitoring tools (Prometheus JMX exporter, Grafana) scraping the Router's JMX port during Router startup/restart; test harnesses that mock Router without stubbing getMetrics(); Router startup races where the admin RPC service is up but the metrics service is not yet registered.","solutions":["Wait for full Router startup (log line 'Router: STARTED' / serviceStart completed) before querying the NameNode JMX beans","Verify the Router process is healthy: check state store connectivity and the router.out/log for failed service init","In tests, mock Router.getMetrics() to return a non-null RBFMetrics before exercising NamenodeBeanMetrics","Retry the JMX/RPC query on a short interval; transient during restarts"],"exampleFix":null,"handlingStrategy":"validation","validationCode":"// Check the Router is in STARTED state before querying federation metrics\nif (router.getServiceStatus() != Service.STATE.STARTED || router.getMetrics() == null) {\n  throw new IllegalStateException(\"Router metrics not ready; retry after startup\");\n}\nNamenodeBeanMetrics nnMetrics = router.getNamenodeMetrics();","typeGuard":null,"tryCatchPattern":"try {\n  RBFMetrics m = nnBeanMetrics.getRBFMetricsSource(); // wrapped getter\n} catch (IOException e) {\n  if (e.getMessage().contains(\"Federated metrics is not initialized\")) {\n    // startup/shutdown window: back off and retry, do not treat as data error\n  } else { throw e; }\n}","preventionTips":["Only scrape Router JMX after the service reports STARTED","Health-check the Router process and state store before running metric-dependent tooling","In tests, always stub Router.getMetrics() to return a non-null RBFMetrics"],"tags":["hdfs","router-based-federation","jmx","metrics","startup-race"],"backgroundTag":"service-not-initialized","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}