grpc/grpc-java · error · ResourceInvalidException

ClusterLoadAssignment has sparse priorities

Error message

ClusterLoadAssignment has sparse priorities

What it means

Priorities in a ClusterLoadAssignment must be dense: every integer from 0 through the maximum observed priority must have at least one locality. If a priority level is skipped (e.g. priorities 0 and 2 exist but 1 does not), the EDS resource is rejected with ResourceInvalidException.

Source

Thrown at xds/src/main/java/io/grpc/xds/XdsEndpointResource.java:149

      LocalityLbEndpoints localityLbEndpoints = structOrError.getStruct();
      int priority = localityLbEndpoints.priority();
      maxPriority = Math.max(maxPriority, priority);
      // Note endpoints with health status other than HEALTHY and UNKNOWN are still
      // handed over to watching parties. It is watching parties' responsibility to
      // filter out unhealthy endpoints. See EnvoyProtoData.LbEndpoint#isHealthy().
      Locality locality =  parseLocality(localityLbEndpointsProto.getLocality());
      localityLbEndpointsMap.put(locality, localityLbEndpoints);
      if (!priorities.containsKey(priority)) {
        priorities.put(priority, new HashSet<>());
      }
      if (!priorities.get(priority).add(locality)) {
        throw new ResourceInvalidException("ClusterLoadAssignment has duplicate locality:"
            + locality + " for priority:" + priority);
      }
    }
    if (priorities.size() != maxPriority + 1) {
      throw new ResourceInvalidException("ClusterLoadAssignment has sparse priorities");
    }

    for (ClusterLoadAssignment.Policy.DropOverload dropOverloadProto
        : assignment.getPolicy().getDropOverloadsList()) {
      dropOverloads.add(parseDropOverload(dropOverloadProto));
    }
    return new EdsUpdate(assignment.getClusterName(), localityLbEndpointsMap, dropOverloads);
  }

  private static Locality parseLocality(io.envoyproxy.envoy.config.core.v3.Locality proto) {
    return Locality.create(proto.getRegion(), proto.getZone(), proto.getSubZone());
  }

  private static DropOverload parseDropOverload(
      io.envoyproxy.envoy.config.endpoint.v3.ClusterLoadAssignment.Policy.DropOverload proto) {
    return DropOverload.create(proto.getCategory(), getRatePerMillion(proto.getDropPercentage()));
  }

View on GitHub (pinned to 64daddc1f3)

Solutions

  1. Renumber priorities to be contiguous starting at 0 in the EDS response
  2. Ensure the management server emits at least one locality for every priority 0..max
  3. Fall back to lower priority count if a tier has no healthy endpoints instead of omitting it
  4. Add control-plane validation for priority density before publishing updates

Example fix

# before
localities:
- {locality: A, priority: 0}
- {locality: B, priority: 2}
# after
localities:
- {locality: A, priority: 0}
- {locality: B, priority: 1}
Defensive patterns

Strategy: validation

Validate before calling

Set<Integer> prios = assignment.getEndpointsList().stream()
    .map(LocalityLbEndpoints::getPriority)
    .collect(Collectors.toSet());
int max = Collections.max(prios);
if (prios.size() != max + 1) throw new IllegalArgumentException("sparse priorities");

Try / catch

try {
  update = XdsEndpointResource.getInstance().parse(args, resource);
} catch (ResourceInvalidException e) {
  logger.warn("EDS rejected: {}", e.getMessage());
}

Prevention

When it happens

Trigger: processClusterLoadAssignment counts distinct priority keys in the priorities map and compares with maxPriority + 1; any gap (sparse set) triggers the error — e.g. localities declared at priority 0 and 2 with none at 1.

Common situations: Control planes removing all localities at a middle priority during failover reconfigurations; generators computing priorities non-contiguously; stale cached endpoint sets partially merged.

Understand the failure class

Background: "Must be a positive integer", "Invalid value", "Unsupported": the invalid-argument-value error family, when a library rejects the value you pass — this error's family across 35 libraries.

Related errors


AI-assisted analysis of grpc/grpc-java@64daddc1f3 (2026-09-08). Data as JSON: /api/errors/c2e6799be39ec6c5. Report an issue: GitHub.