apache/hadoop · error · DiskBalancerException

INTERNAL_ERROR

INTERNAL_ERROR

Error message

Internal error, Unable to create JSON string.

What it means

DiskBalancerException with Result.INTERNAL_ERROR from DiskBalancer.getVolumeNames: serializing the storageID-to-volume-base-path map to JSON via JsonUtil.toJsonString threw an unexpected IOException, which the wrapper converts into this generic internal error. The map itself is built from live FsVolumeSpi references, so in practice the serialization failure indicates an unexpected runtime condition (reflection/IO trouble inside the JSON writer or a concurrent volume state change) rather than a user input problem.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DiskBalancer.java:308

      shutdownExecutor();
    }
  }

  /**
   * Returns a volume ID to Volume base path map.
   *
   * @return Json string of the volume map.
   * @throws DiskBalancerException
   */
  public String getVolumeNames() throws DiskBalancerException {
    lock.lock();
    try {
      checkDiskBalancerEnabled();
      return JsonUtil.toJsonString(getStorageIDToVolumeBasePathMap());
    } catch (DiskBalancerException ex) {
      throw ex;
    } catch (IOException e) {
      throw new DiskBalancerException("Internal error, Unable to " +
          "create JSON string.", e,
          DiskBalancerException.Result.INTERNAL_ERROR);
    } finally {
      lock.unlock();
    }
  }

  /**
   * Returns the current bandwidth.
   *
   * @return string representation of bandwidth.
   * @throws DiskBalancerException
   */
  public long getBandwidth() throws DiskBalancerException {
    lock.lock();
    try {
      checkDiskBalancerEnabled();
      return this.bandwidth;

View on GitHub (pinned to 2add963021)

Solutions

1. Retry the query after a few seconds - transient concurrent volume state usually resolves
2. Check DataNode logs at the same timestamp for the underlying IOException (it is chained as the cause) and address that root cause
3. Verify all volumes are healthy (DataNode JMX volumeFailures == 0) and none are being added/removed mid-query
4. Restart the DataNode if its DiskBalancer state is visibly wedged (query consistently fails with the same cause)

try {
  String volumesJson = diskBalancer.getVolumeNames();
} catch (DiskBalancerException e) {
  if (e.getResult() == DiskBalancerException.Result.INTERNAL_ERROR) {
    // inspect getCause() for the real IOException; retry once after volumes
    // settle; if persistent, restart the DataNode to reset DiskBalancer state
  }
}

When it happens

Trigger: 'hdfs diskbalancer -query <datanode>' (or any code path calling getVolumeNames, e.g. plan generation tooling listing volumes) when JsonUtil.toJsonString throws IOException instead of returning the volume map JSON.

Common situations: Rare; typically surfaces alongside volume failures where the volume map is being mutated while listed, or on partially-initialized DiskBalancer state right after startup/restart. Retry usually succeeds once volumes settle.

Related errors

• No data in ${path}
• Failed to read JSON file ${e}
• Failed to serialize object to JSON
• Bad status:
• Recovery block {b} where it is not under construction.

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/9548946bdfa9182d. Report an issue: GitHub.