apache/beam · error

error decoding document ID: %w

Error message

error decoding document ID: %w

What it means

decodeDocument first decodes each cursor document into a documentID struct to extract the _id used for tracking positions. If that decode fails, the error is wrapped as 'error decoding document ID'. This means a returned document's _id could not be represented by the driver's documentID type (unusual _id types or malformed documents).

Source

Thrown at sdks/go/pkg/beam/io/mongodbio/read.go:312

	ctx context.Context,
	filter bson.M,
) (*mongo.Cursor, error) {
	opts := options.Find().
		SetProjection(fn.projection).
		SetSort(bson.M{"_id": 1})

	cursor, err := fn.collection.Find(ctx, filter, opts)
	if err != nil {
		return nil, fmt.Errorf("error executing find command: %w", err)
	}

	return cursor, nil
}

func decodeDocument(cursor *mongo.Cursor, t reflect.Type) (id any, value any, err error) {
	var docID documentID
	if err := cursor.Decode(&docID); err != nil {
		return nil, nil, fmt.Errorf("error decoding document ID: %w", err)
	}

	out := reflect.New(t).Interface()
	if err := cursor.Decode(out); err != nil {
		return nil, nil, fmt.Errorf("error decoding document: %w", err)
	}

	value = reflect.ValueOf(out).Elem().Interface()

	return docID.ID, value, nil
}

View on GitHub (pinned to 12126d8942)

Solutions

1. Inspect the failing document's _id type; migrate or normalize exotic _id values to supported types (string, ObjectId, int64)
2. Update the mongo-driver to the latest version for broader BSON type support
3. Exclude unsupported documents from the read range via the filter
4. Check driver/server version compatibility if a new BSON type appeared after a server upgrade
5. Wrap reads with retry logic for transient cursor errors

Example fix

// before — collection stores _id as mixed types including arrays
coll.InsertOne(ctx, bson.M{"_id": []int{1,2}, "v": 1})
// after — use a stable scalar id
coll.InsertOne(ctx, bson.M{"_id": primitive.NewObjectID(), "v": 1})

// verify all _id types in the range are scalars the driver supports
cur, _ := coll.Find(ctx, bson.M{"_id": bson.M{"$type": bson.A{"array","regex","javascript"}}})
if n, _ := countLeftovers(cur, ctx); n > 0 { /* migrate or exclude them first */ }

id, _, err := decodeDocument(cursor, t)
if err != nil {
	log.Printf("skipping undecodable document %v: %v", cursor.Current.Lookup("_id"), err)
}

When it happens

Trigger: ProcessElement iterates the cursor and calls decodeDocument; cursor.Decode(&docID) fails — the document's _id uses a BSON type that cannot decode into the internal documentID struct (e.g. binary subtype, very exotic types), or the cursor/stream yields corrupt or interrupted data.

Common situations: Collections with heterogeneous or unusual _id types (e.g. arrays, regex, embedded null bytes); driver version that can't handle a server-inserted BSON type; network corruption or cursor invalidated mid-read; reading system/admin collections with odd documents.

Related errors

• error decoding document: %w
• error encoding BSON: %w
• error decoding BSON: %w
• error decoding buckets: %w
• mongodbio.newReadFn: %v

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/b839b0da7adad6a9. Report an issue: GitHub.