apache/beam · error

error executing bucketAuto aggregation: %w

Error message

error executing bucketAuto aggregation: %w

What it means

getBuckets executes a $bucketAuto aggregation pipeline to partition the collection's _id range into roughly equal-size buckets for splitting. If collection.Aggregate fails to start, the error is wrapped as 'error executing bucketAuto aggregation'. This indicates the aggregation itself was rejected or failed server/network-side.

Source

Thrown at sdks/go/pkg/beam/io/mongodbio/id_range_split.go:120

	pipeline := mongo.Pipeline{
		bson.D{{
			Key:   "$match",
			Value: filter,
		}},
		bson.D{{
			Key: "$bucketAuto",
			Value: bson.M{
				"groupBy": "$_id",
				"buckets": count,
			},
		}},
	}

	opts := options.Aggregate().SetAllowDiskUse(true)

	cursor, err := collection.Aggregate(ctx, pipeline, opts)
	if err != nil {
		return nil, fmt.Errorf("error executing bucketAuto aggregation: %w", err)
	}

	var buckets []bucket
	if err := cursor.All(ctx, &buckets); err != nil {
		return nil, fmt.Errorf("error decoding buckets: %w", err)
	}

	return buckets, nil
}

func idRangesFromBuckets(buckets []bucket, outerRange idRange) []idRange {
	if len(buckets) == 0 {
		return nil
	}

	ranges := make([]idRange, len(buckets))

	for i := 0; i < len(buckets); i++ {

View on GitHub (pinned to 12126d8942)

Solutions

1. Verify the MongoDB server version supports $bucketAuto (3.4+)
2. Grant the connecting user the aggregate privilege on the collection
3. Inspect the wrapped cause for a specific server error (e.g. code 168 for invalid pipeline)
4. Increase allowDiskUse capacity / reduce parallelism, or switch to the splitVector-based split strategy
5. Check network stability and context deadlines during planning

Example fix

// before — server 3.2 lacks $bucketAuto, falls over
splits, err := bucketAutoSplits(ctx, collection, r, numSplits, bundleSize)
// after — use splitVector strategy
cfg := Config{SplitStrategy: SplitVectorStrategy}
splits, err := splitVectorSplits(ctx, db, collection.Name(), r, numSplits, bundleSize)

// $bucketAuto needs MongoDB >= 3.4
var v struct{ Version string }
db.RunCommand(ctx, bson.D{{Key:"buildInfo", Value:1}}).Decode(&v)
major := strings.Split(strings.TrimPrefix(v.Version,"v"), ".")
// require major>3 || (major==3 && minor>=4)

for attempt := 0; attempt < 3; attempt++ {
	buckets, err := getBuckets(ctx, coll, pipeline)
	if err == nil { break }
	time.Sleep(backoff(attempt))
}

When it happens

Trigger: bucketAutoSplits calls getBuckets; the $bucketAuto pipeline (with AllowDiskUse) is rejected by the server — invalid syntax for the server version, authorization failure, memory limits even with disk use, or network/context failure.

Common situations: Older MongoDB servers (<3.4) that lack $bucketAuto; user without aggregate permission on the collection; very large collections exceeding execution limits; context canceled mid-aggregation during split planning.

Understand the failure class

Background: Database query failed: Internal Server Error 500s wrapping SQL, Prisma, and connection failures — what to check first — this error's family across 16 libraries.

Related errors

• error finding document ID to split on: %w
• error executing collStats command: %w
• error decoding buckets: %w
• error executing splitVector command: %w
• Restriction tracker missing.

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/d86b65ed18eca8c8. Report an issue: GitHub.