apache/beam · error

failed to dial server at %v

Error message

failed to dial server at %v

What it means

grpcx.DefaultDial dials a gRPC endpoint with WithBlock and a context timeout; this error wraps any failure from grpc.DialContext, meaning the client could not establish a connection to the server within the timeout. The underlying cause (unreachable, refused, TLS, timeout) is wrapped in the error chain.

Source

Thrown at sdks/go/pkg/beam/util/grpcx/dial.go:39

	"time"

	"github.com/apache/beam/sdks/v2/go/pkg/beam/internal/errors"
	"google.golang.org/grpc"
)

// Dial is a convenience wrapper over grpc.Dial. It can be overridden
// to provide a customized dialing behavior.
var Dial = DefaultDial

// DefaultDial is a dialer that specifies an insecure blocking connection with a timeout.
func DefaultDial(ctx context.Context, endpoint string, timeout time.Duration) (*grpc.ClientConn, error) {
	ctx, cancel := context.WithTimeout(ctx, timeout)
	defer cancel()

	cc, err := grpc.DialContext(ctx, endpoint, grpc.WithInsecure(), grpc.WithBlock(),
		grpc.WithDefaultCallOptions(grpc.MaxCallRecvMsgSize(math.MaxInt32)))
	if err != nil {
		return nil, errors.Wrapf(err, "failed to dial server at %v", endpoint)
	}
	return cc, nil
}

View on GitHub (pinned to 12126d8942)

Solutions

1. Verify the endpoint address and port are correct and the server is listening
2. Check network reachability (telnet/nc) from client to endpoint; fix firewall/DNS/container networking
3. Ensure the server starts before the client dials
4. Inspect the wrapped cause with errors.Unwrap/%+v for the exact transport error

Example fix

// before
cc, err := grpcx.DefaultDial(ctx, "localhost:0") // server on 50051
// after
cc, err := grpcx.DefaultDial(ctx, "localhost:50051")

conn, err := net.DialTimeout("tcp", host, 2*time.Second)
if err != nil {
	return fmt.Errorf("endpoint %s unreachable before dial", host)
}
conn.Close()

cc, err := grpcx.DefaultDial(ctx, endpoint)
if err != nil {
	if strings.Contains(err.Error(), "failed to dial") {
		// retry with backoff or fail fast with clear message
		return errors.Wrapf(err, "server %s not reachable", endpoint)
	}
	return err
}

When it happens

Trigger: Calling grpcx.DefaultDial(ctx, endpoint) (or drivers using it to reach harness/workers) when the server isn't listening, the address/port is wrong, a firewall blocks it, or the connection isn't ready before the default timeout expires.

Common situations: Worker containers can't reach the driver/host endpoint (Docker/K8s networking, wrong hostname), server started after the client, port not published, or DNS resolution failure in a container network.

Understand the failure class

Background: ECONNREFUSED and "connection refused" / "could not connect to server" errors: what they mean and how to fix them — this error's family across 44 libraries.

Related errors

• error connecting to job server at %v: %v
• failed to connect to data service at %v
• Error starting background log thread
• Logging stream terminated unexpectedly before it was closed
• Failed to call Rate Limit Service

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/a16e4fdd3bf4261e. Report an issue: GitHub.