apache/beam · error

expansion failed

Error message

expansion failed

What it means

Raised in xlangx.QueryExpansionService when the retried client.Expand gRPC call to the expansion service ultimately fails. The library wraps the final gRPC error as "expansion failed" after exhausting retry attempts. It means the remote expansion service could not successfully process the ExpansionRequest at the transport/RPC level.

Source

Thrown at sdks/go/pkg/beam/core/runtime/xlangx/expand.go:187

	client := jobpb.NewExpansionServiceClient(conn)

	// Handling ExpansionResponse
	retryOpts := []retry.Option{
		retry.Attempts(maxRetries),
		retry.DelayType(func(n uint, err error, config *retry.Config) time.Duration {
			if n == 0 {
				return time.Second
			}
			return retry.BackOffDelay(n, err, config)
		}),
	}
	var res *jobpb.ExpansionResponse
	err = retry.Do(func() error {
		res, err = client.Expand(ctx, req)
		return err
	}, retryOpts...)
	if err != nil {
		err = errors.Wrap(err, "expansion failed")
		return nil, errors.WithContextf(err, "expanding transform with ExpansionRequest: %v", req)
	}
	if len(res.GetError()) != 0 { // ExpansionResponse includes an error.
		err := errors.New(res.GetError())
		err = errors.Wrap(err, "expansion response error")
		return nil, errors.WithContextf(err, "expanding transform with ExpansionRequest: %v", req)
	}

	return res, nil
}

func startAutomatedJavaExpansionService(gradleTarget string, classpath string) (stopFunc func() error, address string, err error) {
	jarPath, err := expansionx.GetBeamJar(gradleTarget, core.SdkVersion)
	if err != nil {
		return nil, "", err
	}

	if len(classpath) > 0 {

View on GitHub (pinned to 12126d8942)

Solutions

1. Check the expansion service logs for the crash or error returned per attempt
2. Verify service health and connectivity, then re-run the pipeline
3. Increase/adjust retry options if the service is intermittently overloaded
4. Align Beam SDK versions between the Go pipeline and the expansion service (e.g. matching Python Beam version)

Example fix

// before
// service started ad hoc and dies under load
// after
// run expansion service with more memory / stable port and matching Beam version
java -jar beam-sdks-java-expansion-service-<version>.jar 4444

if err := waitForExpansionService(ctx, addr, 5*time.Second); err != nil {
    return err // fail fast if service is down before submitting
}

res, err := xlangx.QueryExpansionService(ctx, params)
if err != nil {
    if isTransient(err) {
        return retryWithBackoff(ctx, func() error { _, err = xlangx.QueryExpansionService(ctx, params); return err })
    }
    return fmt.Errorf("expansion RPC failed permanently: %w", err)
}

When it happens

Trigger: QueryPythonExpansionService/QueryAutomatedExpansionService calls where client.Expand returns an error on every retry: service crashed mid-request, connection dropped, RPC deadline exceeded, or service returned a gRPC error status.

Common situations: Expansion service OOMs or dies while expanding a large transform; network interruption between pipeline and service; service too slow and retries exhausted; incompatible expansion service version rejecting the request.

Understand the failure class

Background: "API request failed": what wrapped HTTP errors from external APIs mean and how to find the real cause — this error's family across 29 libraries.

Related errors

• Failed to call Rate Limit Service
• unable to connect to expansion service at %v
• expansion service error: %s
• Logging stream terminated unexpectedly before it was closed
• Retryable failure

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/18cd1ae79c598f82. Report an issue: GitHub.