googleapis/mcp-toolbox · error

failed to create dataproc session template client: %w

Error message

failed to create dataproc session template client: %w

What it means

Initialize creates a SessionTemplateControllerClient after the batch client succeeds. If this second client cannot be constructed (credentials, endpoint, or network issues), the already-created batchClient is closed and the error is wrapped with this message. The wrapped cause from Google's client library identifies the specific problem.

Source

Thrown at internal/sources/serverlessspark/serverlessspark.go:80

func (r Config) SourceConfigType() string {
	return SourceType
}

func (r Config) Initialize(ctx context.Context, tracer trace.Tracer) (sources.Source, error) {
	ua, err := util.UserAgentFromContext(ctx)
	if err != nil {
		return nil, fmt.Errorf("error in User Agent retrieval: %s", err)
	}
	endpoint := fmt.Sprintf("%s-dataproc.googleapis.com:443", r.Location)
	batchClient, err := dataproc.NewBatchControllerClient(ctx, option.WithEndpoint(endpoint), option.WithUserAgent(ua))
	if err != nil {
		return nil, fmt.Errorf("failed to create dataproc batch client: %w", err)
	}
	sessionTemplateClient, err := dataproc.NewSessionTemplateControllerClient(ctx, option.WithEndpoint(endpoint), option.WithUserAgent(ua))
	if err != nil {
		batchClient.Close()
		return nil, fmt.Errorf("failed to create dataproc session template client: %w", err)
	}
	opsClient, err := longrunning.NewOperationsClient(ctx, option.WithEndpoint(endpoint), option.WithUserAgent(ua))
	if err != nil {
		batchClient.Close()
		sessionTemplateClient.Close()
		return nil, fmt.Errorf("failed to create longrunning client: %w", err)
	}
	sessionClient, err := dataproc.NewSessionControllerClient(ctx, option.WithEndpoint(endpoint), option.WithUserAgent(ua))
	if err != nil {
		batchClient.Close()
		sessionTemplateClient.Close()
		opsClient.Close()
		return nil, fmt.Errorf("failed to create dataproc session client: %w", err)
	}

	s := &Source{
		Config:                r,
		BatchClient:           batchClient,

View on GitHub (pinned to 8cc6e09de2)

Solutions

1. Retry Initialize — transient credential/token fetch failures often resolve; check wrapped cause for details.
2. Apply the same ADC/endpoint/API-enablement checks as the batch client: `gcloud services enable dataproc.googleapis.com`.
3. Check VPC Service Controls / org policies that may allow batch but block session template API calls to `<region>-dataproc.googleapis.com`.
4. Verify network stability and proxy settings to googleapis.com:443.
5. Confirm the service account has dataproc permissions covering session templates (roles/dataproc.editor).

Example fix

// before — org policy blocks the API, retry with valid egress
// after — allow egress to <region>-dataproc.googleapis.com:443 and enable API
gcloud services enable dataproc.googleapis.com --project my-project
export GOOGLE_APPLICATION_CREDENTIALS=/path/to/sa-key.json

// Same pre-flight as the batch client, plus org-policy awareness
// Ensure both Dataproc surfaces are allowed before init:
gcloud services list --enabled --project my-project | grep dataproc.googleapis.com
# If behind VPC Service Controls, confirm the session template API is in the allowed list

src, err := cfg.Initialize(ctx, tracer)
if err != nil && strings.Contains(err.Error(), "session template client") {
    // transient failures are common mid multi-client setup — retry once
    time.Sleep(2 * time.Second)
    src, err = cfg.Initialize(ctx, tracer)
}
if err != nil { return err }

When it happens

Trigger: dataproc.NewSessionTemplateControllerClient(ctx, option.WithEndpoint(...), option.WithUserAgent(ua)) returns an error while NewBatchControllerClient previously succeeded — e.g. transient credential/API failure, endpoint resolution issue, or partially enabled APIs.

Common situations: Same credential/endpoint issues as the batch client but surfacing only here due to transient token fetch failure; region string valid for batch but session template API restricted by org policy; VPC Service Controls blocking the session template endpoint; intermittent network failures during multi-client setup.

Related errors

• failed to create dataproc batch client: %w
• error getting email from ADC: %v
• failed to initialize dataplex client: %w
• failed to create impersonated credentials for %q: %w
• failed to find default Google Cloud credentials with scopes

AI-assisted analysis of googleapis/mcp-toolbox@8cc6e09de2 (2026-09-05). Data as JSON: /api/errors/efbc5ece0a8c7213. Report an issue: GitHub.