What causes "could not close data operations client: %v" in apache/beam?

Inspect prior worker logs — a failed Close after successful writes is usually benign/transient.

could not close data operations client: %v

Error message

could not close data operations client: %v

What it means

In writeFn.Teardown, the Bigtable client's Close() result is checked; a non-nil error is wrapped as "could not close data operations client". This happens after writing completes and usually indicates transport or gRPC shutdown issues.

Source

Thrown at sdks/go/pkg/beam/io/bigtableio/bigtable.go:157

	table *bigtable.Table `json:"-"`
	// Type is the encoded schema type.
	Type beam.EncodedType `json:"type"`
}

func (f *writeFn) Setup(ctx context.Context) error {
	var err error
	f.client, err = bigtable.NewClient(ctx, f.Project, f.InstanceID)
	if err != nil {
		return fmt.Errorf("could not create data operations client: %v", err)
	}

	f.table = f.client.Open(f.TableName)
	return nil
}

func (f *writeFn) Teardown() error {
	if err := f.client.Close(); err != nil {
		return fmt.Errorf("could not close data operations client: %v", err)
	}
	return nil
}

func (f *writeFn) ProcessElement(ctx context.Context, key int, values func(*Mutation) bool) error {

	var mutation Mutation
	for values(&mutation) {

		err := validateMutation(mutation)
		if err != nil {
			return fmt.Errorf("invalid bigtableio.Mutation: %s", err)
		}

		err = f.table.Apply(ctx, mutation.RowKey, getBigtableMutation(mutation))
		if err != nil {
			return fmt.Errorf("could not apply mutation for row key='%s': %v", mutation.RowKey, err)
		}

View on GitHub (pinned to 12126d8942)

Solutions

Inspect prior worker logs — a failed Close after successful writes is usually benign/transient.
Check network stability between workers and Bigtable endpoints.
Retry the job if data was already written; verify no partial-write issues.
If persistent, upgrade the Beam/Google Cloud client libraries, as stale gRPC connections have been fixed over time.

Defensive patterns

Strategy: try-catch

Try / catch

if err := fn.Teardown(); err != nil {
    if strings.Contains(err.Error(), "close data operations client") {
        log.Printf("non-fatal client close error: %v", err) // usually transient
    }
}

Prevention

Treat close errors after successful writes as transient; verify data completeness.
Keep Beam and google-cloud-go client libraries up to date.
Monitor worker network health during shutdown windows.

When it happens

Trigger: worker Teardown where f.client.Close() returns an error — typically gRPC connection teardown failures, lost network mid-job, or a client that was left in a bad state after earlier errors.

Common situations: Workers losing connectivity during shutdown; transient gRPC errors on Close; cascading failures after earlier Apply/Setup errors.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/119fddb006412fb1. Report an issue: GitHub.