alibaba/nacos · critical · NacosException

No available server after {} retries, last tried server: {},

Error message

No available server after {} retries, last tried server: {}, last errMsg: {}

What it means

Thrown by ClientHttpProxy when all server retry attempts are exhausted AND at least one server produced a request exception (requestException != null). The error code is propagated from the last captured exception via requestException.getErrCode(), and the message includes the last server address and error message. This indicates the servers were reachable but returned errors (auth failures, 5xx, etc.).

Source

Thrown at maintainer-client/src/main/java/com/alibaba/nacos/maintainer/client/remote/ClientHttpProxy.java:165

                LOGGER.error("[NACOS Exception] Server address: {}, Error: {}", currentServerAddr,
                    ex.getMessage());
                resultCode = HttpURLConnection.HTTP_INTERNAL_ERROR;
            }
            
            if (isFail(resultCode)) {
                currentServerAddr = serverListManager.genNextServer();
            }
            retryCount--;
            
            try {
                Thread.sleep(100);
            } catch (InterruptedException e) {
                Thread.currentThread().interrupt();
            }
        }
        
        if (null != requestException) {
            throw new NacosException(requestException.getErrCode(),
                "No available server after " + maxRetry + " retries, last tried server: "
                    + currentServerAddr
                    + ", last errMsg: " + requestException.getErrMsg());
        }
        throw new NacosException(NacosException.BAD_GATEWAY,
            "No available server after " + maxRetry + " retries, last tried server: "
                + currentServerAddr);
    }
    
    private String resolveErrorMessage(HttpRestResult<String> result) {
        String responseBody = result.getData();
        if (StringUtils.isNotBlank(responseBody)) {
            try {
                Result<Object> response =
                    JsonUtils.toObj(responseBody, new NacosTypeReference<Result<Object>>() {
                    });
                if (response != null) {
                    String message = response.getMessage();

View on GitHub (pinned to 9b989acdf1)

Solutions

Check the last errMsg in the exception message to identify the root-cause server error and fix it (e.g. refresh credentials, fix auth config).
Verify all server addresses in the server list are correct and reachable.
Increase maxRetry via the maintainer client retry property if the underlying error is transient.
Inspect server-side logs for the matching error at the timestamp of the failure.

Defensive patterns

Strategy: retry

Try / catch

try {
    result = maintainerService.someApiCall(...);
} catch (NacosException e) {
    if (e.getErrCode() == NacosException.BAD_GATEWAY || isTransient(e)) {
        // exponential backoff retry
        Thread.sleep(backoffMillis);
        backoffMillis *= 2;
    } else {
        // inspect e.getErrMsg() for the root cause (auth, 5xx, etc.)
        log.error("Nacos call failed permanently: {}", e.getErrMsg(), e);
        throw e;
    }
}

Prevention

Ensure credentials (username/password or access token) are valid and not expired before making calls.
Monitor server health and keep at least one healthy node in the server list.
Log the full errMsg from the caught exception to diagnose the underlying server-side error.
Configure an appropriate maxRetry value for your network reliability.

When it happens

Trigger: The maintainer client's HTTP proxy loop tries each server in the server list up to maxRetry times. Every attempt threw an exception (e.g. HTTP 403 from invalid credentials, 500 from server error). After retries are exhausted, the last captured exception's code and message are re-thrown.

Common situations: Invalid or expired access token / username-password causing every server to return 403/401. All server nodes are overloaded returning 5xx. Network partition where connections succeed but responses fail. Server in the middle of a rolling restart returning transient errors.

Related errors

AI-assisted analysis of alibaba/nacos@9b989acdf1 (2026-08-14). Data as JSON: /api/errors/fee17fb0c6f4e675. Report an issue: GitHub.