louislam/uptime-kuma · error · Error

All ${errors.length} nodes failed because ${errors.join("; "

Error message

All ${errors.length} nodes failed because ${errors.join("; ")}

What it means

The check() loop iterates every node and calls checkSingleNode; each failure is caught, logged at warn, and appended to an errors array as 'Node N: <message>'. If the loop completes without any success, no single node was reachable, so it throws an aggregate error summarizing the count and each per-node failure joined by '; '. This is the all-nodes-down terminal condition.

Source

Thrown at server/monitor-types/rabbitmq.js:46

            const nodeIndex = i + 1;

            try {
                await this.checkSingleNode(monitor, baseUrl, `${nodeIndex}/${baseUrls.length}`);
                // If checkSingleNode succeeds (doesn't throw), set heartbeat to UP
                heartbeat.status = UP;
                heartbeat.msg =
                    baseUrls.length === 1
                        ? "Node is reachable and there are no alerts in the cluster"
                        : `One of the ${baseUrls.length} nodes is reachable and there are no alerts in the cluster`;
                return;
            } catch (error) {
                log.warn(this.name, `Node ${nodeIndex}: ${error.message}`);
                errors.push(`Node ${nodeIndex}: ${error.message}`);
            }
        }

        // If we reach here, all nodes failed
        throw new Error(`All ${errors.length} nodes failed because ${errors.join("; ")}`);
    }

    /**
     * Check a single RabbitMQ node
     * @param {object} monitor Monitor configuration
     * @param {string} baseUrl Base URL of the RabbitMQ node
     * @param {string} nodeInfo Node index info for logging (e.g., "1/3")
     * @returns {Promise<void>}
     * @throws {Error} If the node check fails
     */
    async checkSingleNode(monitor, baseUrl, nodeInfo) {
        // Without a trailing slash, path in baseUrl will be removed. https://example.com/api -> https://example.com
        let normalizedUrl = baseUrl;
        if (!normalizedUrl.endsWith("/")) {
            normalizedUrl += "/";
        }

        const options = {

View on GitHub (pinned to 6b5ea01557)

Solutions

1. Read each 'Node N: ...' segment; if all share the same cause (e.g. 401), fix that shared cause first.
2. Verify each node's management API responds: curl -u user:pass http://node:15672/api/aliveness-test/%2F.
3. Check credentials, network reachability, and that the rabbitmq_management plugin is enabled on every node.
4. If only some nodes are truly down, consider removing the permanently-unreachable ones from the list.

Example fix

# before
All 3 nodes failed because Node 1: 401 - Unauthorized; Node 2: 401 - Unauthorized; Node 3: 401 - Unauthorized
# after  (correct the shared credential)
Update monitor.rabbitmqUsername / rabbitmqPassword

// Pre-flight: probe each node independently so the aggregate error never fires.
async function probeAllNodes(monitor) {
  const nodes = parseRabbitNodes(monitor.rabbitmqNodes);
  const results = await Promise.allSettled(
    nodes.map(url => checkSingleNode(monitor, url, url))
  );
  if (!results.some(r => r.status === 'fulfilled')) {
    const reasons = results.map((r, i) => `Node ${i+1}: ${r.reason.message}`).join('; ');
    throw new Error(`All ${results.length} nodes failed because ${reasons}`);
  }
}

function allFailed(errors) { return Array.isArray(errors) && errors.length > 0 && errors.length === totalNodes; }

try {
  await rabbitmqMonitor.check(monitor, heartbeat, server);
} catch (e) {
  if (/^All \d+ nodes failed because/.test(e.message)) {
    // surface per-node breakdown; treat as cluster-wide outage, page operator
    heartbeat.status = DOWN;
    heartbeat.msg = e.message;
  }
}

When it happens

Trigger: Every node in baseUrls failed checkSingleNode — each either returned non-200, returned 503 with a reason, timed out, or threw a network error. The message lists all of them so the operator can see the full pattern rather than only the last failure.

Common situations: RabbitMQ cluster fully down, network partition isolating all nodes, wrong credentials applied to every node (each 401/403), or the management plugin disabled on all nodes (each 404).

Related errors

• Request timed out
• ${error.message}
• Invalid Docker response, is it Docker really a daemon?
• Connection to Docker daemon timed out.
• RabbitMQ Nodes must be valid JSON: ${e.message}

AI-assisted analysis of louislam/uptime-kuma@6b5ea01557 (2026-08-12). Data as JSON: /api/errors/b8ae4ef02d6073e9. Report an issue: GitHub.