grpc/grpc-java · error · ResourceInvalidException

Failed to parse Locality Endpoint metadata: ${e.getMessage()

Error message

Failed to parse Locality Endpoint metadata: ${e.getMessage()}

What it means

gRPC's xDS client parses the metadata Struct attached to each LocalityLbEndpoints in an EDS ClusterLoadAssignment resource via MetadataRegistry.parseMetadata. If that Struct cannot be parsed into valid typed metadata, the whole EDS resource is rejected with this ResourceInvalidException (wrapping the underlying reason) and the management server receives a NACK.

Source

Thrown at xds/src/main/java/io/grpc/xds/XdsEndpointResource.java:210

  @VisibleForTesting
  @Nullable
  static StructOrError<LocalityLbEndpoints> parseLocalityLbEndpoints(
      io.envoyproxy.envoy.config.endpoint.v3.LocalityLbEndpoints proto)
      throws ResourceInvalidException {
    // Filter out localities without or with 0 weight.
    if (!proto.hasLoadBalancingWeight() || proto.getLoadBalancingWeight().getValue() < 1) {
      return null;
    }
    if (proto.getPriority() < 0) {
      return StructOrError.fromError("negative priority");
    }

    ImmutableMap<String, Object> localityMetadata;
    MetadataRegistry registry = MetadataRegistry.getInstance();
    try {
      localityMetadata = registry.parseMetadata(proto.getMetadata());
    } catch (ResourceInvalidException e) {
      throw new ResourceInvalidException("Failed to parse Locality Endpoint metadata: "
          + e.getMessage(), e);
    }
    List<Endpoints.LbEndpoint> endpoints = new ArrayList<>(proto.getLbEndpointsCount());
    for (io.envoyproxy.envoy.config.endpoint.v3.LbEndpoint endpoint : proto.getLbEndpointsList()) {
      // The endpoint field of each lb_endpoints must be set.
      // Inside of it: the address field must be set.
      if (!endpoint.hasEndpoint() || !endpoint.getEndpoint().hasAddress()) {
        return StructOrError.fromError("LbEndpoint with no endpoint/address");
      }
      ImmutableMap<String, Object> endpointMetadata;
      try {
        endpointMetadata = registry.parseMetadata(endpoint.getMetadata());
      } catch (ResourceInvalidException e) {
        throw new ResourceInvalidException("Failed to parse Endpoint metadata: "
            + e.getMessage(), e);
      }
      List<java.net.SocketAddress> addresses = new ArrayList<>();
      addresses.add(getInetSocketAddress(endpoint.getEndpoint().getAddress()));

View on GitHub (pinned to 64daddc1f3)

Solutions

1. Inspect the nested cause message (e.getMessage() appended) to find which metadata field failed parsing in MetadataRegistry
2. Fix the metadata Struct in the EDS resource on the xDS management server so it conforms to the gRPC xDS metadata schema
3. Ensure the locality metadata uses supported JSON Struct types (string/number/bool/nested structs) with expected keys
4. Re-push the corrected ClusterLoadAssignment resource so the client ACKs instead of NACKs

Example fix

// before (control plane emits unsupported metadata)
metadata { fields { key: "zone" value { list_value { values { struct_value {} } } } } }
// after
metadata { fields { key: "zone" value { string_value: "us-east-1a" } } }

// Control-plane side: validate locality metadata before sending
MetadataRegistry registry = MetadataRegistry.getInstance();
try {
  registry.parseMetadata(localityProto.getMetadata()); // must not throw
} catch (ResourceInvalidException e) {
  throw new IllegalStateException("Invalid locality metadata: " + e.getMessage(), e);
}

boolean isValidLocalityMetadata(com.google.protobuf.Struct metadata) {
  try {
    MetadataRegistry.getInstance().parseMetadata(metadata);
    return true;
  } catch (ResourceInvalidException e) {
    return false;
  }
}

// Client side: observe NACK reasons via xDS client listeners
xdsClient.addResourceWatcher(listener, new Watcher<EdsUpdate>() {
  @Override public void onError(Status error) {
    logger.log(WARNING, "EDS resource rejected: " + error.getDescription()); // contains 'Failed to parse Locality Endpoint metadata: ...'
  }
  @Override public void onResourceDoesNotExist(String resourceName) {}
  @Override public void onChanged(EdsUpdate update) {}
});

When it happens

Trigger: An EDS ClusterLoadAssignment resource is delivered whose lb_endpoints[i].locality.metadata Struct uses a type or shape MetadataRegistry cannot parse (e.g. wrong metadata key type, unsupported proto Struct encoding, or a registered parser rejects a field). parseLocalityLbEndpoints catches the ResourceInvalidException from parseMetadata and rethrows with this prefix.

Common situations: Control plane (Envoy-based xDS server, Istio, custom management server) emits locality metadata that violates the gRPC metadata convention; a hand-written EDS response contains malformed metadata; a MetadataRegistry custom parser is stricter than what the control plane emits.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

• Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

Related errors

• Failed to parse Endpoint metadata: ${e.getMessage()}
• Failed to parse metadata key: %s, type: %s. Error: %s
• Failed to parse xDS filter metadata for cluster '" + cluster
• Invalid message type: ${unpackedMessage.getClass()}
• Address is not an IP

AI-assisted analysis of grpc/grpc-java@64daddc1f3 (2026-09-08). Data as JSON: /api/errors/f49bdafaae488297. Report an issue: GitHub.