apache/hadoop · error · IOException

Scheduler's refresh-queues failed with the exception : " + S

Error message

Scheduler's refresh-queues failed with the exception : " + StringUtils.stringifyException(e) + "
" + "Scheduler couldn't refresh it's queues with the new configuration properties. Retaining existing configuration throughout the system.

What it means

During refresh-queues, after QueueManager validates the hierarchy, it hands the new queue tree to the configured scheduler via schedulerRefresher.refreshQueues(). If the scheduler rejects the new properties (for example CapacityScheduler finds invalid capacities), the scheduler's exception is stringified into the message and rethrown as this IOException; the whole refresh is rolled back and existing configuration is kept everywhere.

Source

Thrown at hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/QueueManager.java:371

      throw new IOException(MSG_REFRESH_FAILURE_WITH_CHANGE_OF_HIERARCHY);
    }

    /*
     * (2) QueueManager owned properties are validated. Now validate and
     * refresh the properties of scheduler in a single step.
     */
    if (schedulerRefresher != null) {
      try {
        schedulerRefresher.refreshQueues(cp.getRoot().getJobQueueInfo().getChildren());
      } catch (Throwable e) {
        StringBuilder msg =
            new StringBuilder(
                "Scheduler's refresh-queues failed with the exception : "
                    + StringUtils.stringifyException(e));
        msg.append("\n");
        msg.append(MSG_REFRESH_FAILURE_WITH_SCHEDULER_FAILURE);
        LOG.error(msg.toString());
        throw new IOException(msg.toString());
      }
    }

    /*
     * (3) Scheduler has validated and refreshed its queues successfully, now
     * refresh the properties owned by QueueManager
     */

    // First copy the scheduling information recursively into the new
    // queue-hierarchy. This is done to retain old scheduling information. This
    // is done after scheduler refresh and not before it because during refresh,
    // schedulers may wish to change their scheduling info objects too.
    cp.getRoot().copySchedulingInfo(this.root);

    // Now switch roots.
    initialize(cp);

    LOG.info("Queue configuration is refreshed successfully.");

View on GitHub (pinned to 2add963021)

Solutions

Read the exception embedded in the message after 'failed with the exception :' — it names the exact scheduler problem.
Fix the reported scheduler property (make capacities sum to 100, align queue names with mapred-queues.xml, correct max-capacity values).
Re-run 'mr admin -refresh-queues' and confirm it returns cleanly.
If the message is truncated, check the JobTracker log where the full message is also logged at ERROR level.

Defensive patterns

Strategy: try-catch

Validate before calling

// capacity-scheduler sanity check before refresh: capacities must sum to 100
int total = 0;
for (Queue q : newQueues) total += q.getCapacity();
if (total != 100) throw new IOException("Capacities sum to " + total + ", expected 100");

Try / catch

try {
  queueManager.refreshQueues(conf);
} catch (IOException e) {
  if (e.getMessage().contains("Scheduler's refresh-queues failed")) {
    // parse the embedded 'failed with the exception :' text; existing config is retained system-wide
  }
}

Prevention

Keep queue names in capacity-scheduler.xml and mapred-queues.xml in lockstep (generate both from one source).
Verify capacity percentages sum to exactly 100 before deploying.
Set maximum-capacity >= capacity on every queue.

When it happens

Trigger: Refreshing queues with a capacity-scheduler.xml whose queue capacities do not sum to 100, a maximum-capacity below capacity, a queue missing from mapred-queues.xml, or malformed ACL strings in scheduler properties.

Common situations: Editing capacity-scheduler.xml without keeping queue names in sync with mapred-queues.xml; rounding capacity percentages so they total 99 or 101; adding a scheduler queue that the queue config does not define.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/7fbe7544e35091be. Report an issue: GitHub.