quickwit-oss/quickwit · error · anyhow::Error

received message from unassigned partition `{}`. Assigned pa

Error message

received message from unassigned partition `{}`. Assigned partitions: `{{{}}}`

What it means

The Kafka source actor only expects messages from partitions it was explicitly assigned via `AssignPartitions` events. If a message arrives for a partition absent from `assigned_partitions`, the actor treats it as a programming/consistency error and aborts with this message that also lists the currently assigned partitions.

Source

Thrown at quickwit/quickwit-indexing/src/source/kafka_source.rs:315

            partition,
            offset,
            ..
        } = message;

        if let Some(doc) = doc_opt {
            batch.add_doc(doc);
        } else {
            self.state.num_invalid_messages += 1;
        }
        self.state.num_bytes_processed += payload_len;
        self.state.num_messages_processed += 1;

        let partition_id = self
            .state
            .assigned_partitions
            .get(&partition)
            .ok_or_else(|| {
                anyhow::anyhow!(
                    "received message from unassigned partition `{}`. Assigned partitions: \
                     `{{{}}}`",
                    partition,
                    self.state.assigned_partitions.keys().join(", "),
                )
            })?
            .clone();
        let current_position = Position::offset(offset);
        let previous_position = self
            .state
            .current_positions
            .insert(partition, current_position.clone())
            .unwrap_or_else(|| previous_position_for_offset(offset));
        batch
            .checkpoint_delta
            .record_partition_delta(partition_id, previous_position, current_position)
            .context("failed to record partition delta")?;
        Ok(())

View on GitHub (pinned to a39730c5cd)

Solutions

Check consumer group rebalance frequency (session timeouts, `max.poll.interval.ms`) and stabilize membership to reduce revocation races.
Upgrade/verify Quickwit's kafka_source version for fixes to assignment handling during rebalances.
Ensure only one consumer instance per source assignment and no duplicate consumer configs.
Restart the indexing source node to re-establish a clean partition assignment.

Defensive patterns

Strategy: retry

Try / catch

match result {
    Err(e) if e.to_string().contains("received message from unassigned partition") => {
        // restart the source to rebuild assignment; rebalances are transient
    }
    other => other?,
}

Prevention

Tune consumer group settings (session timeout, max.poll.interval) to avoid churn-induced rebalances.
Avoid running duplicate consumers under the same group/source.
Keep broker connectivity stable to prevent spurious revocations.
Monitor consumer group rebalance events alongside indexing logs.

When it happens

Trigger: `process_message` receives a `KafkaEvent::Message` whose partition is not in `self.state.assigned_partitions` — e.g. events from a pre-revocation consumer racing with a `RevokePartitions`, duplicate assignment events, or a misbehaving consumer thread delivering stale records.

Common situations: Kafka consumer group rebalances racing with in-flight polled messages, restarting sources with cached consumer state, or broker/session timeouts triggering rapid partition revocation while buffered messages are still processed.

Understand the failure class

Background: "Invalid state transition" errors: "status must be X, actually Y", "already rejected/charging/uninstalled", "cannot ... while running" — what they mean when a library rejects your call — this error's family across 31 libraries.

Related errors

AI-assisted analysis of quickwit-oss/quickwit@a39730c5cd (2026-09-08). Data as JSON: /api/errors/301b52d0122cf692. Report an issue: GitHub.