vectordotdev/vector · error

MessageStream never returns Ready(None)

Error message

MessageStream never returns Ready(None)

What it means

The main Kafka driver loop polls the StreamConsumer's message stream only to pump client callbacks; messages are consumed in dedicated per-partition tasks. The stream is contracted never to yield None, so that branch is unreachable - firing it means the rdkafka stream terminated, breaking the driver's core assumption.

Source

Thrown at src/sources/kafka.rs:965

    Handle::current().block_on(async move {
        let mut eof: OptionFuture<_> = eof.into();
        let mut stream = consumer.stream();
        loop {
            tokio::select! {
                _ = &mut shutdown => {
                    consumer.context().shutdown();
                    break
                },

                Some(_) = &mut eof => {
                    consumer.context().shutdown();
                    break
                },

                // NB: messages are not received on this thread, however we poll
                // the consumer to serve client callbacks, such as rebalance notifications
                message = stream.next() => match message {
                    None => unreachable!("MessageStream never returns Ready(None)"),
                    Some(Err(error)) => emit!(KafkaReadError { error }),
                    Some(Ok(_msg)) => {
                        unreachable!("Messages are consumed in dedicated tasks for each partition.")
                    }
                },
            }
        }
    });
}

#[allow(clippy::too_many_arguments)]
async fn parse_message(
    msg: BorrowedMessage<'_>,
    decoder: Decoder,
    decompressor: Option<&Decompressor>,
    keys: &'_ Keys,
    out: &mut SourceSender,
    acknowledgements: bool,

View on GitHub (pinned to 3708c39b12)

Solutions

1. Upgrade Vector so the rdkafka version matches the tested behavior
2. Verify the deployment doesn't swap librdkafka (ldd check inside the image)
3. Log with RUST_LOG=rdkafka=trace to see the terminal stream state before the panic
4. Report with versions - stream termination indicates an upstream contract change

// The driver loop's panic ends the source; supervise and restart:
if let Err(je) = tokio::spawn(kafka::run(cfg, shutdown)).await {
    if je.is_panic() { /* log payload, restart with backoff; report upstream contract change */ }
}

When it happens

Trigger: The rdkafka MessageStream ending (consumer context dropped beneath it, or a version where the stream terminates after error/shutdown) while the driver loop is still selecting on it.

Common situations: rdkafka/tokio-util version drift; embeddings that drop the consumer context while the source runs; librdkafka fatal errors ending the stream in newer client versions.

Related errors

• MessageStream never calls Ready(None)
• Messages are consumed in dedicated tasks for each partition.
• Partition consumer finished after completion.
• Partition assignment received after completion.
• Partitions revoked after completion.

AI-assisted analysis of vectordotdev/vector@3708c39b12 (2026-08-20). Data as JSON: /api/errors/0c91fcbeed9b42de. Report an issue: GitHub.