quickwit-oss/quickwit · error

Partition is owned by this indexing pipeline but is not at t

Error message

Partition is owned by this indexing pipeline but is not at the beginning. This should never happen! Please, report on https://github.com/quickwit-oss/quickwit/issues.

What it means

During partition acquisition, a partition that this indexing pipeline already owns must be resumable from its beginning (eof or beginning position). If an owned partition is found at a non-beginning position, the shared state invariant is broken, and acquire_partitions bails, asking for a bug report because this should be impossible by construction.

Source

Thrown at quickwit/quickwit-indexing/src/source/queue_sources/shared_state.rs:157

        let mut shards = Vec::new();
        let mut re_acquired_shards = Vec::new();
        for sub in open_shard_resp.subresponses {
            // we could also just cast the shard_id back to a partition_id
            let partition_id = partitions[sub.subrequest_id as usize].clone();
            let shard = sub.open_shard();
            let position = shard.publish_position_inclusive.clone().unwrap_or_default();
            let is_owned = sub.open_shard().publish_token.as_deref() == Some(publish_token);
            let update_datetime = OffsetDateTime::from_unix_timestamp(shard.update_timestamp)
                .context("Invalid shard update timestamp")?;
            let is_stale =
                OffsetDateTime::now_utc() - update_datetime > self.reacquire_grace_period;
            if position.is_eof() || (is_owned && position.is_beginning()) {
                shards.push((partition_id, position));
            } else if !is_owned && is_stale {
                info!(previous_token = shard.publish_token, "shard re-acquired");
                re_acquired_shards.push(shard.shard_id().clone());
            } else if is_owned && !position.is_beginning() {
                bail!(
                    "Partition is owned by this indexing pipeline but is not at the beginning. This should never happen! Please, report on https://github.com/quickwit-oss/quickwit/issues."
                )
            }
        }

        if re_acquired_shards.is_empty() {
            return Ok(shards);
        }

        // re-acquire shards that have a token that is not the local token
        let acquire_shard_resp = self
            .metastore
            .acquire_shards(AcquireShardsRequest {
                index_uid: Some(self.source_uid.index_uid.clone()),
                source_id: self.source_uid.source_id.clone(),
                shard_ids: re_acquired_shards,
                publish_token: publish_token.to_string(),
            })

View on GitHub (pinned to a39730c5cd)

Solutions

1. File a bug at https://github.com/quickwit-oss/quickwit/issues with logs and the shard/partition state.
2. Reset the indexing pipeline state for the affected index/source so the partition positions are rebuilt from scratch.
3. Check for prior crashes or manual edits of persisted source state that could leave inconsistent positions.

match acquire_partitions(...) {
    Ok(shards) => shards,
    Err(e) => { log::error!("partition ownership invariant broken: {e}"); /* reset source state and resync shard positions */ }
}

When it happens

Trigger: acquire_partitions encounters a shard/partition whose publish token says it is owned by this pipeline, but its stored position is neither EOF nor at the beginning of the stream.

Common situations: Corrupted or stale shard position state (e.g., after an unclean restart or metastore/state divergence); a bug in position bookkeeping for Kinesis shards; mixing state from different pipeline generations.

Understand the failure class

Background: "This is a bug, please report it": internal invariant violations, unreachable panics, and SNH errors explained — this error's family across 47 libraries.

Related errors

• node not found in pending
• OTP logs or traces do not support VRL transforms
• position of a Kafka partition should never be EOF
• position of a Kinesis shard should never be EOF
• unable to sniff region from environment

AI-assisted analysis of quickwit-oss/quickwit@a39730c5cd (2026-09-08). Data as JSON: /api/errors/430e859e24760aca. Report an issue: GitHub.