{"record":{"id":"2e8f6443dad2ef3e","repo":"vectordotdev/vector","slug":"encoding","errorCode":null,"errorMessage":"{}","messagePattern":"\\{\\}","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"src/sinks/util/encoding.rs","lineNumber":58,"sourceCode":"        bytes_written += batch_prefix.len();\n\n        let mut byte_size = telemetry().create_request_count_byte_size();\n\n        for (position, mut event) in events.into_iter().with_position() {\n            self.0.transform(&mut event);\n\n            // Ensure the json size is calculated after any fields have been removed\n            // by the transformer.\n            byte_size.add_event(&event, event.estimated_json_encoded_size_of());\n\n            let mut bytes = BytesMut::new();\n            match (position, encoder.framer()) {\n                (position, Framer::CharacterDelimited(_) | Framer::NewlineDelimited(_))\n                    if position.is_last() =>\n                {\n                    encoder\n                        .serialize(event, &mut bytes)\n                        .map_err(|error| io::Error::new(io::ErrorKind::InvalidData, error))?;\n                }\n                _ => {\n                    encoder\n                        .encode(event, &mut bytes)\n                        .map_err(|error| io::Error::new(io::ErrorKind::InvalidData, error))?;\n                }\n            }\n            write_all(writer, n_events_pending, &bytes)?;\n            bytes_written += bytes.len();\n            n_events_pending -= 1;\n        }\n\n        let batch_suffix = encoder.batch_suffix(is_empty);\n        assert!(n_events_pending == 0);\n        write_all(writer, 0, batch_suffix)?;\n        bytes_written += batch_suffix.len();\n\n        Ok((bytes_written, byte_size))","sourceCodeStart":40,"sourceCodeEnd":76,"githubUrl":"https://github.com/vectordotdev/vector/blob/bdb87aeaa4c4ff27c0ba643c1c77b21bf2ef4013/src/sinks/util/encoding.rs#L40-L76","documentation":"When encoding an event for a sink whose framer is character- or newline-delimited and the event is the last in the batch, Vector calls the encoder's serialize method. If serialization fails (e.g. the event cannot be rendered by the configured codec, such as JSON serialization of non-serializable values), the underlying error is wrapped into an io::Error with ErrorKind::InvalidData and propagated, failing the whole encode_input call. This signals the event payload, not the transport, is at fault.","triggerScenarios":"encode_input is called with position.is_last() and a CharacterDelimited or NewlineDelimited framer, and encoder.serialize(event, ...) returns an error (codec cannot serialize the event, e.g. VRL/JSON serializer rejects the event structure).","commonSituations":"A transform produces events that the sink's encoding codec cannot serialize (e.g. non-string map keys in a text-oriented codec, NaN/Infinity in JSON, schema mismatch when using json_formatter); misconfigured encoding.charset or codec settings; events of unexpected shape after a remap.","solutions":["Inspect the event shape upstream and add a remap/filter transform so events conform to what the sink codec can serialize.","Check the sink's encoding configuration (codec, only_fields, json formatting) and fix invalid options.","Look at the wrapped inner error message in logs (ComponentEventsDropped 'Failed to encode events') for the precise serialization cause.","Update Vector if the codec is known to reject valid input (codec bug)."],"exampleFix":"// before: events with non-serializable fields flow to a json newline-delimited sink\n// after: sanitize events before the sink\n[transforms.sanitize]\n  type = \"remap\"\n  source = '\"\n    . = map_values(.) -> { key: to_string(value) }\n  \"'","handlingStrategy":"validation","validationCode":"// Vector config check (before deploy): ensure events are JSON-serializable\n[transforms.precheck]\n  type = \"remap\"\n  drop_on_error = true\n  source = 'assert(is_object(.), message: \"event must be an object\")'","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Match the sink codec to your event shapes (use json codec for complex events).","Add a remap transform with drop_on_error before encoding-sensitive sinks.","Monitor ComponentEventsDropped<UNINTENTIONAL> logs/metrics."],"tags":["encoding","serialization","sink","invalid-data"],"backgroundTag":"json-serialization-failed","analyzedSha":"bdb87aeaa4c4ff27c0ba643c1c77b21bf2ef4013","analyzedAt":"2026-09-16T02:53:35.741Z","contentChangedAt":"2026-09-16T02:53:35.741Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}