grpc/grpc-go · error · ErrAllPrioritiesRemoved

no priority is provided, all priorities are removed

Error message

no priority is provided, all priorities are removed

What it means

ErrAllPrioritiesRemoved is returned by the priority load balancer's picker when an xDS config update results in zero priorities. The priority LB policy (used in xDS for failover ordering) requires at least one priority; if a config update removes all of them, it sets the gRPC channel to TransientFailure with this error picker, causing all RPCs to fail immediately.

Solutions

Check the xDS CDS/LB policy configuration to ensure at least one priority/locality is configured.
If using failover priorities, verify the primary and all fallback priorities are present in the config.
Inspect the xDS config dump (istioctl proxy-config cluster) for empty or missing priority lists.
Wait for the next xDS update that restores priorities, or rollback the config change.

Example fix

// On the xDS management server, ensure the cluster LB policy has priorities.
// before: config with empty priorities (all removed)
// after (Istio DestinationRule with localityLbSetting):
trafficPolicy:
  loadBalancer:
    simple: ROUND_ROBIN
  localityLbSetting:
    enabled: true
    distribute:
    - from: region/zone/*
      to:
        "region/zone1/*": 100  # <-- at least one priority

Defensive patterns

Strategy: validation

Validate before calling

// Validate xDS priority config has at least one priority.
// On the control plane, ensure the LB policy config includes priorities.
// Check via:
//   istioctl proxy-config cluster <pod> -o json | jq '.. | .priorities?'
// Ensure the output is non-empty.

Try / catch

// RPCs fail with this error when all priorities are removed.
// This is a config error; retry after config is fixed.
err := client.Call(ctx, req)
if err != nil && strings.Contains(err.Error(), "all priorities are removed") {
    log.Error("xDS priority config is empty; check management server CDS/LB policy")
}

Prevention

Ensure xDS cluster/LB policy config always has at least one priority/locality.
Validate xDS config completeness before pushing (use istioctl analyze).
Monitor for xDS config updates that produce empty priority lists.
Keep fallback cluster configurations to avoid total outage during config transitions.

When it happens

Trigger: An xDS CDS/EDS or LB policy config update that produces an empty priorities list. The balancer at balancer.go:182-187 detects len(b.priorities)==0, sets childInUse to empty, and pushes a TransientFailure state with NewErrPicker(ErrAllPrioritiesRemoved). Any RPC attempt picks this error.

Common situations: xDS management server sends a Cluster resource update that removes all localities/endpoints; a failover configuration where all priority levels are deleted; control plane bug or partial config push that zeroes the priority list; transient state during xDS resource deletion before re-adding.

Related errors

AI-assisted analysis of grpc/grpc-go@0c51461d27 (2026-08-11). Data as JSON: /api/errors/3af1874236698b21. Report an issue: GitHub.

Appendix: source

Thrown at internal/xds/balancer/priority/balancer_priority.go:31

 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 *
 */

package priority

import (
	"errors"
	"time"

	"google.golang.org/grpc/balancer"
	"google.golang.org/grpc/connectivity"
)

var (
	// ErrAllPrioritiesRemoved is returned by the picker when there's no priority available.
	ErrAllPrioritiesRemoved = errors.New("no priority is provided, all priorities are removed")
	// DefaultPriorityInitTimeout is the timeout after which if a priority is
	// not READY, the next will be started. It's exported to be overridden by
	// tests.
	DefaultPriorityInitTimeout = 10 * time.Second
)

// syncPriority handles priority after a config update or a child balancer
// connectivity state update. It makes sure the balancer state (started or not)
// is in sync with the priorities (even in tricky cases where a child is moved
// from a priority to another).
//
// It's guaranteed that after this function returns:
//
//	If some child is READY, it is childInUse, and all lower priorities are
//	closed.
//
//	If some child is newly started(in Connecting for the first time), it is
//	childInUse, and all lower priorities are closed.

View on GitHub (pinned to 0c51461d27)