vectordotdev/vector · error · ChunkedGelfDecoderError

Received chunk with message id {message_id} and sequence num

Error message

Received chunk with message id {message_id} and sequence number {sequence_number} has different total chunks values: original total chunks value is {original_total_chunks} and received total chunks value is {received_total_chunks}

What it means

GELF reassembly keys pending chunks by the 8-byte message id and assumes every chunk of a message agrees on the total-chunks count. When a chunk arrives whose total-chunts byte differs from the value recorded when the message state was created, `TotalChunksMismatchSnafu` rejects it, because merging chunks of two different messages would corrupt data. This is effectively a message-id collision or a sender bug detector.

Source

Thrown at lib/codecs/src/decoding/framing/chunked_gelf.rs:407

            // We need to spawn a task that will clear the message state after a certain time
            // otherwise we will have a memory leak due to messages that never complete
            let state = Arc::clone(&self.state);
            let timeout = self.timeout;
            let timeout_handle = tokio::spawn(async move {
                tokio::time::sleep(timeout).await;
                let mut state_lock = state.lock().expect("poisoned lock");
                if state_lock.remove(&message_id).is_some() {
                    warn!(
                        message_id = message_id,
                        timeout_secs = timeout.as_secs_f64(),
                        "Message was not fully received within the timeout window. Discarding it."
                    );
                }
            });
            Box::new(MessageState::new(total_chunks, timeout_handle))
        });

        ensure!(
            message_state.total_chunks == total_chunks,
            TotalChunksMismatchSnafu {
                message_id,
                sequence_number,
                original_total_chunks: message_state.total_chunks,
                received_total_chunks: total_chunks
            }
        );

        if message_state.is_chunk_present(sequence_number) {
            debug!(
                message_id = message_id,
                sequence_number = sequence_number,
                "Received a duplicate chunk. Ignoring it."
            );
            return Ok(None);
        }

View on GitHub (pinned to 99894c8d88)

Solutions

Make the sender use a unique random 64-bit message id per message (as the GELF spec intends) — e.g. UUID bytes or a strong hash of the payload.
If retransmission re-chunks a message, keep the chunk split identical or use a fresh message id for the retransmission.
Grep sender logs for the reported message_id to identify which two messages collided.

Defensive patterns

Strategy: try-catch

Try / catch

Err(e) if e.to_string().contains("different total chunks values") => {
    warn!(error = %e, "GELF message-id collision; dropping chunk");
}

Prevention

Generate message ids with a strong 64-bit random source (UUID prefix) in custom senders.
Never reuse an id across re-chunked retransmissions.
When this error appears, grep sender logs for the message_id to find the colliding producers.

When it happens

Trigger: Two distinct GELF messages generated with the same 8-byte message id but different chunk counts interleave on the wire; or a sender that recomputes/changes the total for retransmitted chunks of the same id.

Common situations: Custom senders deriving message id from a low-entropy value (timestamp seconds, hash of a template) so distinct large messages collide; duplicated/retx paths that split the same id differently after a payload change; multiple sender processes sharing an id scheme.

Related errors

AI-assisted analysis of vectordotdev/vector@99894c8d88 (2026-08-25). Data as JSON: /api/errors/4b438b4448ca4f6b. Report an issue: GitHub.