{"record":{"id":"8068875c0f8b7d6f","repo":"risingwavelabs/risingwave","slug":"cannot-serialize-document-to-vec-u8","errorCode":null,"errorMessage":"cannot serialize Document to Vec<u8>","messagePattern":"cannot serialize Document to Vec<u8>","errorType":"validation","errorClass":"SinkError","httpStatus":null,"severity":"error","filePath":"src/connector/src/sink/encoder/bson.rs","lineNumber":72,"sourceCode":"            self.pk_indices\n                .iter()\n                .map(|&idx| {\n                    let pk_field = &self.schema.fields[idx];\n                    (\n                        pk_field.name.clone(),\n                        datum_to_bson(pk_field, row.datum_at(idx)),\n                    )\n                })\n                .collect::<Document>()\n                .into()\n        }\n    }\n}\n\nimpl SerTo<Vec<u8>> for Document {\n    fn ser_to(self) -> SinkResult<Vec<u8>> {\n        mongodb::bson::to_vec(&self).map_err(|err| {\n            SinkError::Mongodb(anyhow!(err).context(\"cannot serialize Document to Vec<u8>\"))\n        })\n    }\n}\n\nimpl RowEncoder for BsonEncoder {\n    type Output = Document;\n\n    fn encode_cols(\n        &self,\n        row: impl Row,\n        col_indices: impl Iterator<Item = usize>,\n    ) -> SinkResult<Self::Output> {\n        Ok(col_indices\n            .map(|idx| (&self.schema.fields[idx], row.datum_at(idx)))\n            .map(|(field, datum)| (field.name.clone(), datum_to_bson(field, datum)))\n            .collect())\n    }\n","sourceCodeStart":54,"sourceCodeEnd":90,"githubUrl":"https://github.com/risingwavelabs/risingwave/blob/6469eb736d691e8e9b8a419a57edd6429ca77417/src/connector/src/sink/encoder/bson.rs#L54-L90","documentation":"The MongoDB BSON encoder's SerTo<Vec<u8>> serializes a built BSON Document to bytes via mongodb::bson::to_vec; if serialization fails (e.g., unsupported value types, NaN/f64 anomalies, or oversized documents), the error is wrapped as SinkError::Mongodb with the context \"cannot serialize Document to Vec<u8>\". The row cannot be encoded for the MongoDB sink.","triggerScenarios":"Encoding a row into a BSON Document whose contents cannot be BSON-serialized — e.g., invalid UTF-8 strings, floats that BSON rejects, or documents exceeding the 16MB BSON size limit.","commonSituations":"Sinking rows with very large JSONB payloads exceeding BSON document limits, or columns carrying values that map to BSON-incompatible types.","solutions":["Shrink row payloads — truncate or exclude oversized columns so documents stay under the 16MB BSON limit.","Sanitize problematic column values upstream (e.g., remove NaN/invalid values or convert them to strings).","Check the chained inner error (err) to identify the exact field that failed serialization, then fix that column's encoding."],"exampleFix":"// before: huge jsonb column blows the BSON limit\nCREATE SINK mongo_sink FROM mv WITH (connector='mongodb', ...);\n// after: drop/trim the oversized column\nCREATE SINK mongo_sink FROM (SELECT id, LEFT(big_jsonb, 1000000) AS big_jsonb FROM mv) WITH (connector='mongodb', ...);","handlingStrategy":"try-catch","validationCode":"// reject rows whose estimated serialized size approaches the 16MB BSON limit before sinking\nfn fits_bson(doc: &Document) -> bool { mongodb::bson::to_vec(doc).map(|v| v.len() < 16 * 1024 * 1024).unwrap_or(false) }","typeGuard":null,"tryCatchPattern":"match mongodb::bson::to_vec(&doc) { Ok(bytes) => bytes, Err(e) => { log::error!(\"bson ser failed: {e}\"); return Err(SinkError::Mongodb(anyhow!(e).context(\"cannot serialize Document to Vec<u8>\"))); } }","preventionTips":["Bound JSONB/text column sizes upstream (truncate or exclude oversized fields).","Log the chained bson error to identify the offending field.","Test the sink with representative max-size rows."],"tags":["mongodb","bson","serialization","sink"],"backgroundTag":"json-serialization-failed","analyzedSha":"6469eb736d691e8e9b8a419a57edd6429ca77417","analyzedAt":"2026-09-11T21:06:21.487Z","contentChangedAt":"2026-09-11T21:06:21.487Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}