vectordotdev/vector · error

Partition assignment received after completion.

Error message

Partition assignment received after completion.

What it means

After the Kafka consumer reaches ConsumerState::Complete, the callback channel is expected closed, so no further KafkaCallback deliveries can arrive. A PartitionsAssigned callback hit in Complete means assignments were still delivered after completion - an unreachable state machine violation - so it panics instead of corrupting consumption state.

Source

Thrown at src/sources/kafka.rs:835

                        if !exit_eof {
                            debug!("Partition consumer task finished, while not in draining mode.");
                        }
                        state.keep_consuming(drain_deadline)
                    },
                };

                // PartitionConsumerStatus differentiates between a task that exited after
                // being signaled to end, and one that reached the end of its partition and
                // was configured to exit. After the last such task ends, we signal the kafka
                // driver task to shut down the main consumer too. Note this is only used in tests.
                if exit_eof && status == PartitionConsumerStatus::PartitionEOF && partition_consumers.is_empty() {
                    debug!("All partitions have exited or reached EOF.");
                    let _ = eof.take().map(|e| e.send(()));
                }
            },
            Some(callback) = callbacks.recv() => match callback {
                KafkaCallback::PartitionsAssigned(mut assigned_partitions, done) => match consumer_state {
                    ConsumerState::Complete => unreachable!("Partition assignment received after completion."),
                    ConsumerState::Draining(_) => error!("Partition assignment received while draining revoked partitions, maybe an invalid assignment."),
                    ConsumerState::Consuming(ref consumer_state) => {
                        let acks = consumer.context().acknowledgements;
                        for tp in assigned_partitions.drain(0..) {
                            let topic = tp.0.as_str();
                            let partition = tp.1;
                            match consumer.split_partition_queue(topic, partition) { Some(pq) => {
                                debug!("Consuming partition {}:{}.", &tp.0, tp.1);
                                let (end_tx, handle) = consumer_state.consume_partition(&mut partition_consumers, tp.clone(), Arc::clone(&consumer), pq, acks, exit_eof);
                                abort_handles.insert(tp.clone(), handle);
                                end_signals.insert(tp, end_tx);
                            } _ => {
                                warn!("Failed to get queue for assigned partition {}:{}.", &tp.0, tp.1);
                            }}
                        }
                        // ensure this is retained until all individual queues are set up
                        drop(done);
                    }

View on GitHub (pinned to 3708c39b12)

Solutions

1. Upgrade Vector - completion/callback-close ordering received fixes
2. Reduce shutdown/rebalance overlap: generous graceful_shutdown_timeout and staggered instance restarts
3. Capture the callback sequence in debug logs (assignment arriving after 'All partitions have exited') and report it
4. Review exit_eof usage for that workload - continuous consumption avoids the EOF-completion path entirely

// Cannot be pre-validated; supervise the source task and restart on panic:
if let Err(je) = tokio::spawn(kafka::run(cfg, shutdown)).await {
    if je.is_panic() { /* log JoinError payload, alert, restart with backoff */ }
}

When it happens

Trigger: A rebalance assignment racing the transition to Complete: the driver finalized (shutdown drain done, or EOF-mode completion) while the rdkafka rebalance callback thread still delivered an assignment; also possible when the callback channel wasn't closed on a given version's completion path.

Common situations: Shutdown coinciding with cluster rebalances (scaling events, broker leader changes, k8s pod evictions); sticky assignor churn; Vector/rdkafka version drift changing callback delivery timing.

Related errors

• Partitions revoked after completion.
• Partition consumer finished after completion.
• Shutdown received after completion.
• MessageStream never calls Ready(None)
• Drain deadline received after completion.

AI-assisted analysis of vectordotdev/vector@3708c39b12 (2026-08-20). Data as JSON: /api/errors/5ecbcdaa21f52ecb. Report an issue: GitHub.