What causes "err" in apache/beam?

Verify the MongoDB URI, credentials, and network reachability from where the code runs

err

Error message

err

What it means

mongodbio's newIDRangeRestriction panics when CountDocuments fails while merging the ID-range filter with the user filter to size a restriction. The document count is required for the restriction's splitting logic, so the error cannot be returned through the normal pipeline path and is raised as a panic. It usually reflects a connectivity, authentication, or bad-filter problem against MongoDB.

Source

Thrown at sdks/go/pkg/beam/io/mongodbio/id_range_restriction.go:57

type idRangeRestriction struct {
	IDRange      idRange
	CustomFilter bson.M
	Count        int64
}

// newIDRangeRestriction creates a new idRangeRestriction and counts the documents within the ID
// range that match the custom filter.
func newIDRangeRestriction(
	ctx context.Context,
	collection *mongo.Collection,
	idRange idRange,
	filter bson.M,
) idRangeRestriction {
	mergedFilter := mergeFilters(idRange.Filter(), filter)

	count, err := collection.CountDocuments(ctx, mergedFilter)
	if err != nil {
		panic(err)
	}

	return idRangeRestriction{
		IDRange:      idRange,
		CustomFilter: filter,
		Count:        count,
	}
}

// Filter returns a bson.M filter based on the restriction's ID range and custom filter.
func (r idRangeRestriction) Filter() bson.M {
	idFilter := r.IDRange.Filter()
	return mergeFilters(idFilter, r.CustomFilter)
}

// mergeFilters merges the ID filter and the custom filter into a single bson.M filter.
func mergeFilters(idFilter bson.M, customFilter bson.M) bson.M {
	if len(idFilter) == 0 {

View on GitHub (pinned to 12126d8942)

Solutions

Verify the MongoDB URI, credentials, and network reachability from where the code runs
Validate the bson.M filter operators against your server's MongoDB version
Check mongod logs and collection existence
Wrap the call and recover() in pipeline construction code to convert the panic into a logged error

Example fix

// before
res := mongodbio.CreateInitialRestriction(...) // panics on count error
// after
func() {
    defer func() {
        if r := recover(); r != nil {
            log.Fatalf("mongodbio restriction failed: %v", r)
        }
    }()
    res := mongodbio.CreateInitialRestriction(...)
    _ = res
}()

Defensive patterns

Strategy: try-catch

Validate before calling

if err := client.Ping(ctx, nil); err != nil { return fmt.Errorf("mongo unreachable: %w", err) }
// validate filter operators against server version before building restrictions

Try / catch

defer func() { if r := recover(); r != nil { log.Fatalf("mongodbio restriction: %v", r) } }()

Prevention

Ping MongoDB during pipeline setup before creating restrictions
Keep filter documents simple and version-compatible
Monitor mongod logs for auth/timeout errors correlated with pipeline starts

When it happens

Trigger: Calling CreateInitialRestriction (or building restrictions from ID ranges) when the MongoDB collection is unreachable, credentials are wrong, the server times out, or the merged BSON filter contains invalid operators like $foo that the server rejects.

Common situations: MongoDB behind VPC not reachable from the worker; wrong URI/password; version mismatch where a filter operator is unsupported; collection dropped mid-pipeline setup.

Understand the failure class

Background: Database query failed: Internal Server Error 500s wrapping SQL, Prisma, and connection failures — what to check first — this error's family across 16 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/a8af093154077299. Report an issue: GitHub.