apache/beam · error

error decoding document: %w

Error message

error decoding document: %w

What it means

After extracting the _id, decodeDocument decodes the full document into a newly allocated value of the user's element type via reflection. If cursor.Decode(out) fails, the error is wrapped as 'error decoding document'. This means the MongoDB document's fields cannot be mapped onto the Go type the user configured for the read (missing/bad bson tags, type mismatches).

Source

Thrown at sdks/go/pkg/beam/io/mongodbio/read.go:317

		SetSort(bson.M{"_id": 1})

	cursor, err := fn.collection.Find(ctx, filter, opts)
	if err != nil {
		return nil, fmt.Errorf("error executing find command: %w", err)
	}

	return cursor, nil
}

func decodeDocument(cursor *mongo.Cursor, t reflect.Type) (id any, value any, err error) {
	var docID documentID
	if err := cursor.Decode(&docID); err != nil {
		return nil, nil, fmt.Errorf("error decoding document ID: %w", err)
	}

	out := reflect.New(t).Interface()
	if err := cursor.Decode(out); err != nil {
		return nil, nil, fmt.Errorf("error decoding document: %w", err)
	}

	value = reflect.ValueOf(out).Elem().Interface()

	return docID.ID, value, nil
}

View on GitHub (pinned to 12126d8942)

Solutions

1. Compare the failing document's field types against the target struct's bson tags and fix the struct or use bson.M/raw decoding
2. Normalize the collection's schema so each field has a consistent BSON type
3. Add a custom decode hook / UnmarshalBSON on the target type for flexible types
4. Update the mongo-driver for better type coercion and clearer decode errors
5. Filter out incompatible documents in the read query while migrating

Example fix

// before
type Event struct {
	Count string `bson:"count"` // stored as int in Mongo
}
// after
type Event struct {
	Count int `bson:"count"`
}

// dry-run decode one document into the target type before launching
probe := reflect.New(t).Interface()
if err := coll.FindOne(ctx, filter).Decode(probe); err != nil {
	return fmt.Errorf("element type incompatible: %w", err)
}

out := reflect.New(t).Interface()
if err := cursor.Decode(out); err != nil {
	return nil, nil, fmt.Errorf("row %v: %w", docID.ID, err)
}

When it happens

Trigger: cursor.Decode(out) fails while reading: a document field's BSON type does not match the target Go struct field (e.g. string into int), a required bson tag is wrong, a field is a BSON type the Go type cannot accept, or the cursor yields an invalid document mid-stream.

Common situations: Evolving schemas: a field stored as int historically but string for newer documents; bson tag typos after refactor; reading a collection written by another service with different field types; time.Time vs primitive.DateTime mismatches.

Related errors

• error decoding document ID: %w
• error encoding BSON: %w
• error decoding BSON: %w
• error decoding buckets: %w
• decoding a *%v

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/3d175276afd18d9c. Report an issue: GitHub.