alibaba/nacos · error · NacosException

503

503

Error message

Runtime Endpoint result exceeds 1000 natural keys

What it means

A single RAD discovery or snapshot query produced more than 1000 (MAX_RUNTIME_ENDPOINTS) distinct endpoint natural keys. The AgentRuntimeRegistryService.validateCapacity method (line 355) throws this NacosException (code OVER_THRESHOLD, effectively HTTP 503) to protect the server from unbounded response sizes. Natural keys are unique (namespaceId, agentName, protocol, endpoint URI/transport) combinations after deduplication.

Source

Thrown at ai/src/main/java/com/alibaba/nacos/ai/service/agent/runtime/AgentRuntimeRegistryService.java:356

        current.setBindings(new ArrayList<RuntimeVersionBinding>(bindings));
        current.setEnabled(current.getEnabled() || contribution.getEnabled());
        current.setHealthy(current.getHealthy() || contribution.getHealthy());
        current.setState(runtimeState(current.getEnabled(), current.getHealthy()));
    }
    
    private RuntimeEndpointState runtimeState(boolean enabled, boolean healthy) {
        if (!enabled) {
            return RuntimeEndpointState.DISABLED;
        }
        if (!healthy) {
            return RuntimeEndpointState.UNHEALTHY;
        }
        return RuntimeEndpointState.AVAILABLE;
    }
    
    private void validateCapacity(int size) throws NacosException {
        if (size > MAX_RUNTIME_ENDPOINTS) {
            throw new NacosException(NacosException.OVER_THRESHOLD,
                "Runtime Endpoint result exceeds " + MAX_RUNTIME_ENDPOINTS
                    + " natural keys");
        }
    }
    
    private Service composeService(String namespaceId, String agentName, String protocol) {
        return Service.newService(namespaceId, Constants.Agent.AGENT_ENDPOINT_GROUP,
            RadServiceNameComposer.compose(agentName, protocol));
    }
    
    private void validateReadIdentity(String namespaceId, String agentName, String protocol,
        String version) {
        AgentValidationUtils.validateNamespaceId(namespaceId);
        AgentValidationUtils.validateAgentName(agentName);
        AgentValidationUtils.validateProtocol(protocol);
        if (version != null) {
            AgentValidationUtils.validateVersion(version);
        }

View on GitHub (pinned to 9b989acdf1)

Solutions

Reduce the number of distinct endpoint URIs for this agent/protocol to at most 1000 by consolidating behind load balancers or shared endpoints.
If the scale is legitimate, discuss raising MAX_RUNTIME_ENDPOINTS with maintainers — but consider the memory and response-size implications.
Partition endpoints across multiple agent names or protocols if the Agent serves diverse endpoint sets.
Audit registrations to detect and remove stale or duplicate endpoint URIs.

Defensive patterns

Strategy: validation

Validate before calling

// Before registering, estimate the total unique endpoint count
int estimatedKeys = batch.getEndpoints().stream()
    .map(e -> EndpointNaturalKey.of(ns, name, proto, e))
    .collect(Collectors.toSet()).size();
if (estimatedKeys > 1000) {
    throw new IllegalStateException("Endpoint batch would exceed 1000 natural keys");
}

Type guard

public static boolean isWithinCapacity(int naturalKeyCount) {
    return naturalKeyCount <= 1000;
}

Try / catch

try {
    return registry.getRuntimeEndpointSet(ns, name, proto, versions);
} catch (NacosException e) {
    if (e.getErrCode() == NacosException.OVER_THRESHOLD) {
        // return a capped/paginated result or advise the caller to narrow the query
        logger.warn("Endpoint capacity exceeded for {}/{}/{}", ns, name, proto);
    }
    throw e;
}

Prevention

Consolidate endpoints behind shared URIs/load balancers instead of registering each instance separately.
Monitor the natural-key count per agent/protocol and alert before it approaches 1000.
Partition very large endpoint sets across multiple agent names or protocols.

When it happens

Trigger: Calling getRuntimeEndpointSnapshot or getRuntimeEndpointSet for an agent/protocol whose registered instances collectively define more than 1000 unique endpoint natural keys. This can happen with many publishers each registering distinct endpoints.

Common situations: A large-scale Agent deployment with thousands of endpoint URIs; accidental endpoint proliferation from clients generating unique URIs per registration; a configuration error that registers every micro-instance as a separate endpoint instead of load-balancing behind fewer URIs.

Related errors

AI-assisted analysis of alibaba/nacos@9b989acdf1 (2026-08-14). Data as JSON: /api/errors/598e930acc5a25ef. Report an issue: GitHub.