vectordotdev/vector · error · ChunkedGelfDecoderError::TotalChunksMismatch

Received chunk with message id

Error message

Received chunk with message id {message_id} and sequence number {sequence_number} has different total chunks values: original total chunks value is {original_total_chunks} and received total chunks value is {received_total_chunks}

What it means

When decoding a chunked GELF message (RFC chunked GELF over UDP), each chunk carries a message id and a total chunk count. This error is thrown by decode_chunk when a chunk with the same message id declares a total_chunks value different from the one recorded when the message's first chunk was seen. A mismatch means the sender's chunks cannot be reassembled into one message.

Solutions

1. Fix the producer to generate a unique message id per message (e.g. full md5 hash of the payload or a proper random 64-bit value).
2. Drop/flush the stale pending message on the Vector side by reducing the GELF reassembly timeout so conflicting sequences don't coexist.
3. If ids are truncated prefixes, increase id length so collisions don't occur.
4. Check for multiple senders sharing the same message id space and namespace them per host.

Example fix

// before (producer)
let id = payload_hash[..8].to_vec();
// after
let id = full_md5_hash[..8].to_vec(); // id derived from full message, unique per message

// producer-side check before chunking
fn assert_unique_id(ids: &[[u8; 8]], seen: &mut HashSet<[u8; 8]>) -> bool {
    ids.iter().all(|id| seen.insert(*id))
}

fn is_chunk_consistent(state_total: u8, chunk_total: u8) -> bool {
    state_total == chunk_total
}

When it happens

Trigger: Sending multiple GELF chunk sequences that reuse the same 8-byte/64-bit message id but with differing total chunk counts within the reassembly timeout window; a buggy or colliding producer generating non-unique message ids; chunks from unrelated messages colliding after id truncation (e.g. md5/prefix ids).

Common situations: High-volume syslog/GELF UDP senders where id generation collides; producers that truncate a longer id (like a 64-bit prefix of md5) causing duplicates; multiple hosts behind NAT reusing sequential ids.

Related errors

• Invalid chunk header with less than 10 bytes: 0x
• Received chunk with message id
• Received chunk with message id
• Buffered payload limit of
• Pending messages limit of

AI-assisted analysis of vectordotdev/vector@bdb87aeaa4 (2026-09-16). Data as JSON: /api/errors/4b438b4448ca4f6b. Report an issue: GitHub.

Appendix: source

Thrown at lib/codecs/src/decoding/framing/chunked_gelf.rs:523

                    .get(&message_id)
                    .is_some_and(|message| message.timeout_task.id() == timeout_task_id);
                if owns_message && pending.remove(message_id).is_some() {
                    warn!(
                        message_id = message_id,
                        timeout_secs = timeout.as_secs_f64(),
                        "Message was not fully received within the timeout window. Discarding it."
                    );
                }
            });
            pending.messages.insert(
                message_id,
                Box::new(MessageState::new(total_chunks, timeout_handle)),
            );
        }

        let message_state = pending.messages.get(&message_id).expect("entry must exist");

        ensure!(
            message_state.total_chunks == total_chunks,
            TotalChunksMismatchSnafu {
                message_id,
                sequence_number,
                original_total_chunks: message_state.total_chunks,
                received_total_chunks: total_chunks
            }
        );

        if message_state.is_chunk_present(sequence_number) {
            debug!(
                message_id = message_id,
                sequence_number = sequence_number,
                "Received a duplicate chunk. Ignoring it."
            );
            return Ok(None);
        }

View on GitHub (pinned to bdb87aeaa4)