googleapis/mcp-toolbox · error

failed to create dataproc client: %w

Error message

failed to create dataproc client: %w

What it means

Thrown by the Dataproc source's Config.Initialize when dataproc.NewClusterControllerClient fails to construct the ClusterController gRPC client. This wraps errors from credential resolution, endpoint dialing, or transport creation. Any prior failure prevents the source from initializing at all.

Source

Thrown at internal/sources/dataproc/dataproc.go:74

	Name    string `yaml:"name" validate:"required"`
	Type    string `yaml:"type" validate:"required"`
	Project string `yaml:"project" validate:"required"`
	Region  string `yaml:"region" validate:"required"`
}

func (r Config) SourceConfigType() string {
	return SourceType
}

func (r Config) Initialize(ctx context.Context, tracer trace.Tracer) (sources.Source, error) {
	ua, err := util.UserAgentFromContext(ctx)
	if err != nil {
		return nil, fmt.Errorf("error in User Agent retrieval: %s", err)
	}
	endpoint := fmt.Sprintf("%s-dataproc.googleapis.com:443", r.Region)
	client, err := dataproc.NewClusterControllerClient(ctx, option.WithEndpoint(endpoint), option.WithUserAgent(ua))
	if err != nil {
		return nil, fmt.Errorf("failed to create dataproc client: %w", err)
	}
	opsClient, err := longrunning.NewOperationsClient(ctx, option.WithEndpoint(endpoint), option.WithUserAgent(ua))
	if err != nil {
		client.Close()
		return nil, fmt.Errorf("failed to create longrunning client: %w", err)
	}
	jobClient, err := dataproc.NewJobControllerClient(ctx, option.WithEndpoint(endpoint), option.WithUserAgent(ua))
	if err != nil {
		client.Close()
		opsClient.Close()
		return nil, fmt.Errorf("failed to create dataproc job client: %w", err)
	}

	s := &Source{
		Config:    r,
		Client:    client,
		OpsClient: opsClient,
		JobClient: jobClient,

View on GitHub (pinned to 8cc6e09de2)

Solutions

1. Verify credentials (GOOGLE_APPLICATION_CREDENTIALS or workload identity) are valid and present.
2. Check the region in the config is correct; the endpoint is built as `<region>-dataproc.googleapis.com:443`.
3. Enable the Dataproc API and ensure network/firewall/proxy allows HTTPS to the endpoint.
4. Inspect the wrapped cause (%w) for the exact gRPC/credential error.

Example fix

// before
region: "us-central11" // invalid

// after
region: "us-central1"

// validate region and credentials before creating the source
validRegions := regexp.MustCompile(`^[a-z]+-[a-z]+\d+$`)
if !validRegions.MatchString(cfg.Region) {
    return fmt.Errorf("invalid GCP region: %q", cfg.Region)
}
if os.Getenv("GOOGLE_APPLICATION_CREDENTIALS") == "" {
    return errors.New("GCP credentials not configured")
}

null

src, err := cfg.Initialize(ctx, tracer)
if err != nil {
    if strings.Contains(err.Error(), "failed to create dataproc client") {
        // check credentials/network, possibly retry with backoff
    }
    return err
}

When it happens

Trigger: Initialize called with invalid/missing Application Default Credentials, unreachable endpoint `<region>-dataproc.googleapis.com:443`, invalid region producing a bad endpoint, or proxy/firewall blocking gRPC traffic.

Common situations: Typo in the configured region (e.g., 'us-central1a'), no credentials in a container, corporate proxy blocking gRPC, Dataproc API disabled on the project.

Related errors

• failed to create dataproc job client: %w
• failed to create Dataplex client for project %q: %w
• failed to create longrunning client: %w
• failed to initialize dataplex client: %w
• failed to create BigQuery client for project %q: %w

AI-assisted analysis of googleapis/mcp-toolbox@8cc6e09de2 (2026-09-05). Data as JSON: /api/errors/201a70a7de0a3035. Report an issue: GitHub.