{"record":{"id":"39ed1171105bd0d4","repo":"prestodb/presto","slug":"no-nodes-available-39ed11","errorCode":"NO_NODES_AVAILABLE","errorMessage":"No nodes available to run query","messagePattern":"No nodes available to run query","errorType":"error_code","errorClass":"PrestoException","httpStatus":null,"severity":"error","filePath":"presto-main-base/src/main/java/com/facebook/presto/execution/scheduler/nodeSelection/TopologyAwareNodeSelector.java","lineNumber":161,"sourceCode":"    public SplitPlacementResult computeAssignments(Set<Split> splits, List<RemoteTask> existingTasks)\n    {\n        NodeMap nodeMap = this.nodeMap.get().get();\n        Multimap<InternalNode, Split> assignment = HashMultimap.create();\n        NodeAssignmentStats assignmentStats = new NodeAssignmentStats(nodeTaskMap, nodeMap, existingTasks);\n\n        int[] topologicCounters = new int[topologicalSplitCounters.size()];\n        Set<NetworkLocation> filledLocations = new HashSet<>();\n        Set<InternalNode> blockedExactNodes = new HashSet<>();\n        boolean splitWaitingForAnyNode = false;\n\n        NodeProvider nodeProvider = nodeMap.getNodeProvider(maxPreferredNodes);\n        for (Split split : splits) {\n            SplitWeight splitWeight = split.getSplitWeight();\n            if (split.getNodeSelectionStrategy() == HARD_AFFINITY) {\n                List<InternalNode> candidateNodes = selectExactNodes(nodeMap, split.getPreferredNodes(nodeProvider), includeCoordinator);\n                if (candidateNodes.isEmpty()) {\n                    log.debug(\"No nodes available to schedule %s. Available nodes %s\", split, nodeMap.getActiveNodes());\n                    throw new PrestoException(NO_NODES_AVAILABLE, \"No nodes available to run query\");\n                }\n                InternalNode chosenNode = bestNodeSplitCount(splitWeight, candidateNodes.iterator(), minCandidates, maxPendingSplitsWeightPerTask, assignmentStats);\n                if (chosenNode != null) {\n                    assignment.put(chosenNode, split);\n                    assignmentStats.addAssignedSplit(chosenNode, splitWeight);\n                }\n                // Exact node set won't matter, if a split is waiting for any node\n                else if (!splitWaitingForAnyNode) {\n                    blockedExactNodes.addAll(candidateNodes);\n                }\n                continue;\n            }\n\n            InternalNode chosenNode = null;\n            int depth = networkLocationSegmentNames.size();\n            int chosenDepth = 0;\n            Set<NetworkLocation> locations = new HashSet<>();\n            for (HostAddress host : split.getPreferredNodes(nodeProvider)) {","sourceCodeStart":143,"sourceCodeEnd":179,"githubUrl":"https://github.com/prestodb/presto/blob/55bb57d202de3b926896fa966c2c4a44c779634e/presto-main-base/src/main/java/com/facebook/presto/execution/scheduler/nodeSelection/TopologyAwareNodeSelector.java#L143-L179","documentation":"TopologyAwareNodeSelector.computeAssignments throws NO_NODES_AVAILABLE when a HARD_AFFINITY split's preferred nodes, resolved through topology-aware node selection, yield an empty candidate set - i.e. none of the split's preferred (e.g. HDFS datanode/locality-preferred) nodes are active in the cluster. The topology-aware scheduler maps split host preferences through network topology; if the mapped nodes are all down or excluded (including the coordinator being excluded), the split cannot be placed and the query fails fast.","triggerScenarios":"A split with HARD_AFFINITY (bucketed execution, connector-preferred localities like HDFS block locations) whose preferredNodes after topology resolution via selectExactNodes yield no active nodes; preferred nodes decommissioned or crashed; include-coordinator=false and only the coordinator matches the locality.","commonSituations":"HDFS datanodes hosting the data blocks are down while running topology-aware scheduling with bucketed execution; rack/node topology configuration (node-scheduler.network-topology settings) misconfigured so locality resolves to nonexistent hosts; decommissioning workers that host hot data; autoscaling shrinks the cluster below the split's locality requirements.","solutions":["Bring the preferred worker nodes back online or restore the connector's underlying storage nodes (e.g. HDFS datanodes).","Set node-scheduler.include-coordinator=true to widen the eligible node set.","Check topology configuration (network-topology segments/file) so split hosts map to real cluster nodes.","Disable bucketed/locality-affinity execution for the affected connector (e.g. hive.bucketed-execution=false) or rewrite data to distribute blocks across live nodes.","Inspect coordinator logs 'No nodes available to schedule <split>' to identify which preferred nodes were missing."],"exampleFix":"// before (etc/config.properties)\nnode-scheduler.include-coordinator=false\n// HDFS locality-preferred nodes all down -> NO_NODES_AVAILABLE\n\n// after\nnode-scheduler.include-coordinator=true\n// plus: repair/replace dead HDFS datanodes hosting the splits","handlingStrategy":"try-catch","validationCode":"// Verify the split's preferred nodes are alive before topology-aware assignment\nList<InternalNode> preferred = selectExactNodes(nodeMap, split.getPreferredNodes(nodeProvider), includeCoordinator);\nif (preferred.isEmpty()) {\n    throw new IllegalStateException(\"Preferred nodes for split are not active: \" + split);\n}","typeGuard":"private static boolean hasLivePreferredNodes(Split split, NodeMap nodeMap, NodeProvider provider, boolean includeCoordinator) {\n    return !selectExactNodes(nodeMap, split.getPreferredNodes(provider), includeCoordinator).isEmpty();\n}","tryCatchPattern":"try {\n    return topologyAwareNodeSelector.computeAssignments(splits, existingTasks);\n} catch (PrestoException e) {\n    if (e.getErrorCode() == StandardErrorCode.NO_NODES_AVAILABLE.toErrorCode().getCode()) {\n        // degrade to simple node selection or wait for preferred nodes to rejoin\n        return simpleNodeSelector.computeAssignments(splits, existingTasks);\n    }\n    throw e;\n}","preventionTips":["Set node-scheduler.include-coordinator=true as a safety valve for locality mismatches.","Monitor HDFS datanode / storage-node health; replace dead nodes hosting hot data.","Validate network topology configuration (segments file) maps split hosts to actual cluster nodes.","Avoid HARD_AFFINITY (bucketed) execution on clusters with frequent node churn/autoscaling."],"tags":["scheduler","topology","hard-affinity","no-workers","presto"],"backgroundTag":"no-nodes-available","analyzedSha":"55bb57d202de3b926896fa966c2c4a44c779634e","analyzedAt":"2026-09-04T12:50:26.162Z","contentChangedAt":"2026-09-04T12:50:26.162Z","schemaVersion":2},"datasetVersion":"2026-09-11T21:17:09.523Z"}