{"record":{"id":"a3fadaa76cd128dd","repo":"apache/hadoop","slug":"fsdatasetspi-has-not-been-initialized","errorCode":null,"errorMessage":"FsDatasetSpi has not been initialized","messagePattern":"FsDatasetSpi has not been initialized","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java","lineNumber":976,"sourceCode":"            DFS_DATANODE_MAX_SLOWDISKS_TO_EXCLUDE_DEFAULT : Integer.parseInt(newVal));\n        result = Integer.toString(maxSlowDisksToExclude);\n        diskMetrics.setMaxSlowDisksToExclude(maxSlowDisksToExclude);\n      }\n      LOG.info(\"RECONFIGURE* changed {} to {}\", property, newVal);\n      return result;\n    } catch (IllegalArgumentException e) {\n      throw new ReconfigurationException(property, newVal, getConf().get(property), e);\n    }\n  }\n\n  private String reconfDfsUsageParameters(String property, String newVal)\n      throws ReconfigurationException {\n    String result = null;\n    try {\n      LOG.info(\"Reconfiguring {} to {}\", property, newVal);\n      if (data == null) {\n        LOG.debug(\"FsDatasetSpi has not been initialized.\");\n        throw new IOException(\"FsDatasetSpi has not been initialized\");\n      }\n      if (property.equals(FS_DU_INTERVAL_KEY)) {\n        long interval = (newVal == null ? FS_DU_INTERVAL_DEFAULT :\n            Long.parseLong(newVal));\n        result = Long.toString(interval);\n        List<FsVolumeImpl> volumeList = data.getVolumeList();\n        for (FsVolumeImpl fsVolume : volumeList) {\n          Map<String, BlockPoolSlice> blockPoolSlices = fsVolume.getBlockPoolSlices();\n          for (BlockPoolSlice value : blockPoolSlices.values()) {\n            value.updateDfsUsageConfig(interval, null, null);\n          }\n        }\n      } else if (property.equals(FS_GETSPACEUSED_JITTER_KEY)) {\n        long jitter = (newVal == null ? FS_GETSPACEUSED_JITTER_DEFAULT :\n            Long.parseLong(newVal));\n        result = Long.toString(jitter);\n        List<FsVolumeImpl> volumeList = data.getVolumeList();\n        for (FsVolumeImpl fsVolume : volumeList) {","sourceCodeStart":958,"sourceCodeEnd":994,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java#L958-L994","documentation":"reconfDfsUsageParameters live-applies fs.du.interval and fs.getspaceused.classname by pushing new settings into every FsVolumeImpl's BlockPoolSlice; it requires the DataNode's FsDatasetSpi (the 'data' field) to exist. If the reconfig request lands while data == null — DN still starting, dataset failed to build, or already shut down — it throws IOException(\"FsDatasetSpi has not been initialized\"), which the method's own catch then wraps into a ReconfigurationException.","triggerScenarios":"Reconfiguring fs.du.interval or fs.getspaceused.classname immediately after process start (racing volume formatting/fencing), on a DN whose FsDataset failed to initialize (bad permissions, failed volumes), or during shutdown while a reconfig driver keeps targeting the node.","commonSituations":"Automation fires reconfig calls as soon as the DN port opens instead of after NN registration; DN startup aborted half-way leaving data null; reconfig storms during rolling upgrades hitting a node mid-restart.","solutions":["Wait until the DN reports to the NameNode (hdfs dfsadmin -report shows it live) and resubmit the reconfiguration","Inspect DN logs for the underlying FsDataset creation failure (permissions, incompatible VERSION, failed volumes) and fix that first","For fs.du.interval / fs.getspaceused.classname prefer static hdfs-site.xml changes plus a rolling restart over live reconfig"],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":"// Only submit usage reconfigs once the DN is fully up:\n// 1) 'hdfs dfsadmin -report' lists the DN as live, or\n// 2) curl http://dn:9864/jmx shows DataNodeActivity beans with heartbeat counters > 0","typeGuard":null,"tryCatchPattern":"try {\n  dn.reconfigurePropertyImpl(key, newVal); // fs.du.interval / fs.getspaceused.classname\n} catch (ReconfigurationException e) {\n  if (e.getCause() instanceof IOException\n      && String.valueOf(e.getCause().getMessage()).contains(\"FsDatasetSpi\")) {\n    // lifecycle race: DN starting or shutting down — retry after it reports to the NN\n  }\n}","preventionTips":["Sequence reconfig tooling after the NN reports the DN as live, not after the port opens","Prefer restart-time config for fs.du.interval and fs.getspaceused.classname","Treat this error as a retryable lifecycle race, not a bad value"],"tags":["hadoop","hdfs","datanode","reconfiguration","initialization","race-condition"],"backgroundTag":"component-not-initialized","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-23T01:17:44.959Z"}