vectordotdev/vector · error

Shutdown received after completion.

Error message

Shutdown received after completion.

What it means

KafkaCallback::ShuttingDown is rdkafka's pre-shutdown notification, handled in Consuming (start drain) and Draining (error + close callbacks) states. In Complete no callbacks are expected at all; receiving one means the kafka client thread signaled shutdown after the driver already finalized - unreachable, so it panics.

Source

Thrown at src/sources/kafka.rs:880

                        d.keep_draining(drain_deadline)
                    },
                    ConsumerState::Consuming(state) => {
                        let (deadline, mut state) = state.begin_drain(max_drain_ms, drain, false);

                        for tp in revoked_partitions.drain(0..) {
                            match end_signals.remove(&tp) { Some(end) => {
                                debug!("Revoking partition {}:{}", &tp.0, tp.1);
                                state.revoke_partition(tp, end);
                            } _ => {
                                debug!("Consumer task for partition {}:{} already finished.", &tp.0, tp.1);
                            }}
                        }

                        state.keep_draining(deadline)
                    }
                },
                KafkaCallback::ShuttingDown(drain) => (drain_deadline, consumer_state) = match consumer_state {
                    ConsumerState::Complete => unreachable!("Shutdown received after completion."),
                    // Shutting down is just like a full assignment revoke, but we also close the
                    // callback channels, since we don't expect additional assignments or rebalances
                    ConsumerState::Draining(state) => {
                        // NB: This would only happen if the task driving the kafka client is
                        // not handling shutdown signals; otherwise this is unreachable code
                        error!("Kafka client handled a shutdown signal while a rebalance was in progress.");
                        callbacks.close();
                        state.keep_draining(drain_deadline)
                    },
                    ConsumerState::Consuming(state) => {
                        callbacks.close();
                        let (deadline, mut state) = state.begin_drain(max_drain_ms, drain, true);
                        if let Ok(tpl) = consumer.assignment() {
                            // TODO  workaround for https://github.com/fede1024/rust-rdkafka/issues/681
                            if tpl.capacity() == 0 {
                                return;
                            }
                            tpl.elements()

View on GitHub (pinned to 3708c39b12)

Solutions

1. Upgrade Vector (shutdown callback ordering fixes)
2. If using exit_eof, verify the workload actually needs it - continuous consumption avoids this completion path
3. Reproduce with debug logs and report the callback ordering
4. Check linked librdkafka consistency in the image: ldd $(which vector) | grep rdkafka

// Process-level containment; the panic kills the source task by design:
//   systemd: Restart=on-failure, Environment=RUST_BACKTRACE=1
if let Err(je) = tokio::spawn(kafka::run(cfg, shutdown)).await {
    if je.is_panic() { /* log, restart with backoff; report if recurring */ }
}

When it happens

Trigger: The rdkafka client-level shutdown callback firing after the driver's own completion transition (e.g. EOF mode completed the consumer and the late client shutdown signal arrives); duplicate delivery of the shutting-down callback after channel close in a given rdkafka version.

Common situations: exit_eof workflows finishing exactly as the client shuts down; rapid restart cycles; rdkafka/librdkafka version mismatches in the deployment image.

Related errors

• Partition consumer finished after completion.
• Partition assignment received after completion.
• Partitions revoked after completion.
• Drain deadline received after completion.
• MessageStream never calls Ready(None)

AI-assisted analysis of vectordotdev/vector@3708c39b12 (2026-08-20). Data as JSON: /api/errors/3ed3b2f6c8d3de75. Report an issue: GitHub.