vectordotdev/vector · error

{}

Error message

{}

What it means

When encoding an event for a sink whose framer is character- or newline-delimited and the event is the last in the batch, Vector calls the encoder's serialize method. If serialization fails (e.g. the event cannot be rendered by the configured codec, such as JSON serialization of non-serializable values), the underlying error is wrapped into an io::Error with ErrorKind::InvalidData and propagated, failing the whole encode_input call. This signals the event payload, not the transport, is at fault.

Solutions

1. Inspect the event shape upstream and add a remap/filter transform so events conform to what the sink codec can serialize.
2. Check the sink's encoding configuration (codec, only_fields, json formatting) and fix invalid options.
3. Look at the wrapped inner error message in logs (ComponentEventsDropped 'Failed to encode events') for the precise serialization cause.
4. Update Vector if the codec is known to reject valid input (codec bug).

Example fix

// before: events with non-serializable fields flow to a json newline-delimited sink
// after: sanitize events before the sink
[transforms.sanitize]
  type = "remap"
  source = '"
    . = map_values(.) -> { key: to_string(value) }
  "'

// Vector config check (before deploy): ensure events are JSON-serializable
[transforms.precheck]
  type = "remap"
  drop_on_error = true
  source = 'assert(is_object(.), message: "event must be an object")'

When it happens

Trigger: encode_input is called with position.is_last() and a CharacterDelimited or NewlineDelimited framer, and encoder.serialize(event, ...) returns an error (codec cannot serialize the event, e.g. VRL/JSON serializer rejects the event structure).

Common situations: A transform produces events that the sink's encoding codec cannot serialize (e.g. non-string map keys in a text-oriented codec, NaN/Infinity in JSON, schema mismatch when using json_formatter); misconfigured encoding.charset or codec settings; events of unexpected shape after a remap.

Understand the failure class

Background: "JSON serialization failed", "not JSON serializable", "Failed to serialize": why JSON marshaling errors happen and how to fix them — this error's family across 46 libraries.

Related errors

• {}
• {}
• a sink must always have an external resource
• building HTTP request failed unexpectedly
• Could not convert duration to JSON

AI-assisted analysis of vectordotdev/vector@bdb87aeaa4 (2026-09-16). Data as JSON: /api/errors/2e8f6443dad2ef3e. Report an issue: GitHub.

Appendix: source

Thrown at src/sinks/util/encoding.rs:58

        bytes_written += batch_prefix.len();

        let mut byte_size = telemetry().create_request_count_byte_size();

        for (position, mut event) in events.into_iter().with_position() {
            self.0.transform(&mut event);

            // Ensure the json size is calculated after any fields have been removed
            // by the transformer.
            byte_size.add_event(&event, event.estimated_json_encoded_size_of());

            let mut bytes = BytesMut::new();
            match (position, encoder.framer()) {
                (position, Framer::CharacterDelimited(_) | Framer::NewlineDelimited(_))
                    if position.is_last() =>
                {
                    encoder
                        .serialize(event, &mut bytes)
                        .map_err(|error| io::Error::new(io::ErrorKind::InvalidData, error))?;
                }
                _ => {
                    encoder
                        .encode(event, &mut bytes)
                        .map_err(|error| io::Error::new(io::ErrorKind::InvalidData, error))?;
                }
            }
            write_all(writer, n_events_pending, &bytes)?;
            bytes_written += bytes.len();
            n_events_pending -= 1;
        }

        let batch_suffix = encoder.batch_suffix(is_empty);
        assert!(n_events_pending == 0);
        write_all(writer, 0, batch_suffix)?;
        bytes_written += batch_suffix.len();

        Ok((bytes_written, byte_size))

View on GitHub (pinned to bdb87aeaa4)