dgraph-io/dgraph · error

unable to reach the Alpha that served that task

Error message

unable to reach the Alpha that served that task

What it means

The Alpha that served the task was found in membership (addr resolved), but conn.GetPools().Get(addr) failed to establish/get a connection pool to it; the error is wrapped with 'unable to reach the Alpha that served that task'. This is a network/connectivity failure to a specific cluster peer.

Source

Thrown at worker/queue.go:62

	}

	// Find the Alpha with the required Raft ID.
	var addr string
	for _, group := range groups().state.GetGroups() {
		for _, member := range group.GetMembers() {
			if member.GetId() == raftId {
				addr = member.GetAddr()
			}
		}
	}
	if addr == "" {
		return nil, fmt.Errorf("the Alpha that served that task is not available")
	}

	// Send the request to the Alpha.
	pool, err := conn.GetPools().Get(addr)
	if err != nil {
		return nil, errors.Wrapf(err, "unable to reach the Alpha that served that task")
	}
	client := pb.NewWorkerClient(pool.Get())
	return client.TaskStatus(ctx, req)
}

// TaskStatus retrieves metadata for a given task ID.
func (*grpcWorker) TaskStatus(ctx context.Context, req *pb.TaskStatusRequest,
) (*pb.TaskStatusResponse, error) {
	taskId := req.GetTaskId()
	meta, err := Tasks.get(taskId)
	if err != nil {
		return nil, err
	}

	resp := &pb.TaskStatusResponse{TaskMeta: meta.uint64()}
	return resp, nil
}

View on GitHub (pinned to 759e242be6)

Solutions

1. Retry after a short delay — the Alpha may be restarting or reconnecting
2. Verify the Alpha process is running and its gRPC port is reachable from this node (ping/nc)
3. Check membership records vs actual addresses (stale ZK/Zero entries) and correct them
4. Confirm TLS and ACL settings match across all Alphas

Example fix

// before
meta, _ := pb.TaskStatusOverNetwork(ctx, req)
// after
var meta *pb.TaskStatusResponse
var err error
for i := 0; i < 5; i++ {
    meta, err = pb.TaskStatusOverNetwork(ctx, req)
    if err == nil { break }
    time.Sleep(time.Duration(1<<i) * time.Second)
}

// Pre-flight reachability check
conn, err := net.DialTimeout("tcp", addr, 2*time.Second)
if err != nil { return fmt.Errorf("alpha %s unreachable: %w", addr, err) }
conn.Close()

meta, err := worker.TaskStatusOverNetwork(ctx, req)
var nerr net.Error
if err != nil && (errors.As(err, &nerr) || strings.Contains(err.Error(), "unable to reach")) {
    time.Sleep(backoff)
    meta, err = worker.TaskStatusOverNetwork(ctx, req)
}

When it happens

Trigger: Calling TaskStatusOverNetwork when the target Alpha at addr is down, restarting, unreachable due to network issues, or listening on a different port than the membership record says.

Common situations: The serving Alpha crashed after starting a backup; firewall or Kubernetes service blocking the internal gRPC port; stale address after a pod rescheduled to a new IP; TLS/ACL configuration mismatch between nodes.

Understand the failure class

Background: ECONNREFUSED and "connection refused" / "could not connect to server" errors: what they mean and how to fix them — this error's family across 44 libraries.

Related errors

• cannot retrieve predicate information
• http fetch: %v
• failed to connect to endpoint [%s]: %w
• failed to disable drain mode: %v
• failed to start external snapshot stream for group %d: %w

AI-assisted analysis of dgraph-io/dgraph@759e242be6 (2026-09-01). Data as JSON: /api/errors/5d249d5e5ce414ac. Report an issue: GitHub.