prestodb/presto · error · PrestoException

NO_NODES_AVAILABLE

NO_NODES_AVAILABLE

Error message

No nodes available to run query

What it means

TopologyAwareNodeSelector.computeAssignments throws NO_NODES_AVAILABLE when a HARD_AFFINITY split's preferred nodes, resolved through topology-aware node selection, yield an empty candidate set - i.e. none of the split's preferred (e.g. HDFS datanode/locality-preferred) nodes are active in the cluster. The topology-aware scheduler maps split host preferences through network topology; if the mapped nodes are all down or excluded (including the coordinator being excluded), the split cannot be placed and the query fails fast.

Solutions

1. Bring the preferred worker nodes back online or restore the connector's underlying storage nodes (e.g. HDFS datanodes).
2. Set node-scheduler.include-coordinator=true to widen the eligible node set.
3. Check topology configuration (network-topology segments/file) so split hosts map to real cluster nodes.
4. Disable bucketed/locality-affinity execution for the affected connector (e.g. hive.bucketed-execution=false) or rewrite data to distribute blocks across live nodes.
5. Inspect coordinator logs 'No nodes available to schedule <split>' to identify which preferred nodes were missing.

Example fix

// before (etc/config.properties)
node-scheduler.include-coordinator=false
// HDFS locality-preferred nodes all down -> NO_NODES_AVAILABLE

// after
node-scheduler.include-coordinator=true
// plus: repair/replace dead HDFS datanodes hosting the splits

// Verify the split's preferred nodes are alive before topology-aware assignment
List<InternalNode> preferred = selectExactNodes(nodeMap, split.getPreferredNodes(nodeProvider), includeCoordinator);
if (preferred.isEmpty()) {
    throw new IllegalStateException("Preferred nodes for split are not active: " + split);
}

private static boolean hasLivePreferredNodes(Split split, NodeMap nodeMap, NodeProvider provider, boolean includeCoordinator) {
    return !selectExactNodes(nodeMap, split.getPreferredNodes(provider), includeCoordinator).isEmpty();
}

try {
    return topologyAwareNodeSelector.computeAssignments(splits, existingTasks);
} catch (PrestoException e) {
    if (e.getErrorCode() == StandardErrorCode.NO_NODES_AVAILABLE.toErrorCode().getCode()) {
        // degrade to simple node selection or wait for preferred nodes to rejoin
        return simpleNodeSelector.computeAssignments(splits, existingTasks);
    }
    throw e;
}

When it happens

Trigger: A split with HARD_AFFINITY (bucketed execution, connector-preferred localities like HDFS block locations) whose preferredNodes after topology resolution via selectExactNodes yield no active nodes; preferred nodes decommissioned or crashed; include-coordinator=false and only the coordinator matches the locality.

Common situations: HDFS datanodes hosting the data blocks are down while running topology-aware scheduling with bucketed execution; rack/node topology configuration (node-scheduler.network-topology settings) misconfigured so locality resolves to nonexistent hosts; decommissioning workers that host hot data; autoscaling shrinks the cluster below the split's locality requirements.

Related errors

• NO_NODES_AVAILABLE
• NO_NODES_AVAILABLE
• NODE_SELECTION_NOT_SUPPORTED
• A null map must have zero entries
• Access control ' ' is already registered

AI-assisted analysis of prestodb/presto@55bb57d202 (2026-09-04). Data as JSON: /api/errors/39ed1171105bd0d4. Report an issue: GitHub.

Appendix: source

Thrown at presto-main-base/src/main/java/com/facebook/presto/execution/scheduler/nodeSelection/TopologyAwareNodeSelector.java:161

    public SplitPlacementResult computeAssignments(Set<Split> splits, List<RemoteTask> existingTasks)
    {
        NodeMap nodeMap = this.nodeMap.get().get();
        Multimap<InternalNode, Split> assignment = HashMultimap.create();
        NodeAssignmentStats assignmentStats = new NodeAssignmentStats(nodeTaskMap, nodeMap, existingTasks);

        int[] topologicCounters = new int[topologicalSplitCounters.size()];
        Set<NetworkLocation> filledLocations = new HashSet<>();
        Set<InternalNode> blockedExactNodes = new HashSet<>();
        boolean splitWaitingForAnyNode = false;

        NodeProvider nodeProvider = nodeMap.getNodeProvider(maxPreferredNodes);
        for (Split split : splits) {
            SplitWeight splitWeight = split.getSplitWeight();
            if (split.getNodeSelectionStrategy() == HARD_AFFINITY) {
                List<InternalNode> candidateNodes = selectExactNodes(nodeMap, split.getPreferredNodes(nodeProvider), includeCoordinator);
                if (candidateNodes.isEmpty()) {
                    log.debug("No nodes available to schedule %s. Available nodes %s", split, nodeMap.getActiveNodes());
                    throw new PrestoException(NO_NODES_AVAILABLE, "No nodes available to run query");
                }
                InternalNode chosenNode = bestNodeSplitCount(splitWeight, candidateNodes.iterator(), minCandidates, maxPendingSplitsWeightPerTask, assignmentStats);
                if (chosenNode != null) {
                    assignment.put(chosenNode, split);
                    assignmentStats.addAssignedSplit(chosenNode, splitWeight);
                }
                // Exact node set won't matter, if a split is waiting for any node
                else if (!splitWaitingForAnyNode) {
                    blockedExactNodes.addAll(candidateNodes);
                }
                continue;
            }

            InternalNode chosenNode = null;
            int depth = networkLocationSegmentNames.size();
            int chosenDepth = 0;
            Set<NetworkLocation> locations = new HashSet<>();
            for (HostAddress host : split.getPreferredNodes(nodeProvider)) {

View on GitHub (pinned to 55bb57d202)