risingwavelabs/risingwave · error · SinkError

cannot serialize Document to Vec<u8>

Error message

cannot serialize Document to Vec<u8>

What it means

The MongoDB BSON encoder's SerTo<Vec<u8>> serializes a built BSON Document to bytes via mongodb::bson::to_vec; if serialization fails (e.g., unsupported value types, NaN/f64 anomalies, or oversized documents), the error is wrapped as SinkError::Mongodb with the context "cannot serialize Document to Vec<u8>". The row cannot be encoded for the MongoDB sink.

Source

Thrown at src/connector/src/sink/encoder/bson.rs:72

            self.pk_indices
                .iter()
                .map(|&idx| {
                    let pk_field = &self.schema.fields[idx];
                    (
                        pk_field.name.clone(),
                        datum_to_bson(pk_field, row.datum_at(idx)),
                    )
                })
                .collect::<Document>()
                .into()
        }
    }
}

impl SerTo<Vec<u8>> for Document {
    fn ser_to(self) -> SinkResult<Vec<u8>> {
        mongodb::bson::to_vec(&self).map_err(|err| {
            SinkError::Mongodb(anyhow!(err).context("cannot serialize Document to Vec<u8>"))
        })
    }
}

impl RowEncoder for BsonEncoder {
    type Output = Document;

    fn encode_cols(
        &self,
        row: impl Row,
        col_indices: impl Iterator<Item = usize>,
    ) -> SinkResult<Self::Output> {
        Ok(col_indices
            .map(|idx| (&self.schema.fields[idx], row.datum_at(idx)))
            .map(|(field, datum)| (field.name.clone(), datum_to_bson(field, datum)))
            .collect())
    }

View on GitHub (pinned to 6469eb736d)

Solutions

1. Shrink row payloads — truncate or exclude oversized columns so documents stay under the 16MB BSON limit.
2. Sanitize problematic column values upstream (e.g., remove NaN/invalid values or convert them to strings).
3. Check the chained inner error (err) to identify the exact field that failed serialization, then fix that column's encoding.

Example fix

// before: huge jsonb column blows the BSON limit
CREATE SINK mongo_sink FROM mv WITH (connector='mongodb', ...);
// after: drop/trim the oversized column
CREATE SINK mongo_sink FROM (SELECT id, LEFT(big_jsonb, 1000000) AS big_jsonb FROM mv) WITH (connector='mongodb', ...);

// reject rows whose estimated serialized size approaches the 16MB BSON limit before sinking
fn fits_bson(doc: &Document) -> bool { mongodb::bson::to_vec(doc).map(|v| v.len() < 16 * 1024 * 1024).unwrap_or(false) }

match mongodb::bson::to_vec(&doc) { Ok(bytes) => bytes, Err(e) => { log::error!("bson ser failed: {e}"); return Err(SinkError::Mongodb(anyhow!(e).context("cannot serialize Document to Vec<u8>"))); } }

When it happens

Trigger: Encoding a row into a BSON Document whose contents cannot be BSON-serialized — e.g., invalid UTF-8 strings, floats that BSON rejects, or documents exceeding the 16MB BSON size limit.

Common situations: Sinking rows with very large JSONB payloads exceeding BSON document limits, or columns carrying values that map to BSON-incompatible types.

Understand the failure class

Background: "JSON serialization failed", "not JSON serializable", "Failed to serialize": why JSON marshaling errors happen and how to fix them — this error's family across 46 libraries.

Related errors

• config deserialization error: {e}
• config deserialization error: {e}
• config deserialization error: {e}
• Avro error: {0}
• sending bulk write command failed, database: {}

AI-assisted analysis of risingwavelabs/risingwave@6469eb736d (2026-09-11). Data as JSON: /api/errors/8068875c0f8b7d6f. Report an issue: GitHub.