{"record":{"id":"430e859e24760aca","repo":"quickwit-oss/quickwit","slug":"partition-is-owned-by-this-indexing-pipeline-but-i","errorCode":null,"errorMessage":"Partition is owned by this indexing pipeline but is not at the beginning. This should never happen! Please, report on https://github.com/quickwit-oss/quickwit/issues.","messagePattern":"Partition is owned by this indexing pipeline but is not at the beginning\\. This should never happen! Please, report on https://github\\.com/quickwit-oss/quickwit/issues\\.","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"quickwit/quickwit-indexing/src/source/queue_sources/shared_state.rs","lineNumber":157,"sourceCode":"        let mut shards = Vec::new();\n        let mut re_acquired_shards = Vec::new();\n        for sub in open_shard_resp.subresponses {\n            // we could also just cast the shard_id back to a partition_id\n            let partition_id = partitions[sub.subrequest_id as usize].clone();\n            let shard = sub.open_shard();\n            let position = shard.publish_position_inclusive.clone().unwrap_or_default();\n            let is_owned = sub.open_shard().publish_token.as_deref() == Some(publish_token);\n            let update_datetime = OffsetDateTime::from_unix_timestamp(shard.update_timestamp)\n                .context(\"Invalid shard update timestamp\")?;\n            let is_stale =\n                OffsetDateTime::now_utc() - update_datetime > self.reacquire_grace_period;\n            if position.is_eof() || (is_owned && position.is_beginning()) {\n                shards.push((partition_id, position));\n            } else if !is_owned && is_stale {\n                info!(previous_token = shard.publish_token, \"shard re-acquired\");\n                re_acquired_shards.push(shard.shard_id().clone());\n            } else if is_owned && !position.is_beginning() {\n                bail!(\n                    \"Partition is owned by this indexing pipeline but is not at the beginning. This should never happen! Please, report on https://github.com/quickwit-oss/quickwit/issues.\"\n                )\n            }\n        }\n\n        if re_acquired_shards.is_empty() {\n            return Ok(shards);\n        }\n\n        // re-acquire shards that have a token that is not the local token\n        let acquire_shard_resp = self\n            .metastore\n            .acquire_shards(AcquireShardsRequest {\n                index_uid: Some(self.source_uid.index_uid.clone()),\n                source_id: self.source_uid.source_id.clone(),\n                shard_ids: re_acquired_shards,\n                publish_token: publish_token.to_string(),\n            })","sourceCodeStart":139,"sourceCodeEnd":175,"githubUrl":"https://github.com/quickwit-oss/quickwit/blob/a39730c5cdcd1a4fe798403737ae293999ea21f8/quickwit/quickwit-indexing/src/source/queue_sources/shared_state.rs#L139-L175","documentation":"During partition acquisition, a partition that this indexing pipeline already owns must be resumable from its beginning (eof or beginning position). If an owned partition is found at a non-beginning position, the shared state invariant is broken, and acquire_partitions bails, asking for a bug report because this should be impossible by construction.","triggerScenarios":"acquire_partitions encounters a shard/partition whose publish token says it is owned by this pipeline, but its stored position is neither EOF nor at the beginning of the stream.","commonSituations":"Corrupted or stale shard position state (e.g., after an unclean restart or metastore/state divergence); a bug in position bookkeeping for Kinesis shards; mixing state from different pipeline generations.","solutions":["File a bug at https://github.com/quickwit-oss/quickwit/issues with logs and the shard/partition state.","Reset the indexing pipeline state for the affected index/source so the partition positions are rebuilt from scratch.","Check for prior crashes or manual edits of persisted source state that could leave inconsistent positions."],"exampleFix":null,"handlingStrategy":"fallback","validationCode":null,"typeGuard":null,"tryCatchPattern":"match acquire_partitions(...) {\n    Ok(shards) => shards,\n    Err(e) => { log::error!(\"partition ownership invariant broken: {e}\"); /* reset source state and resync shard positions */ }\n}","preventionTips":["Avoid unclean shutdowns/crash-loops that can corrupt persisted shard positions.","Monitor for repeated occurrences — it signals a bug; report with logs.","Keep pipeline generation/publish-token bookkeeping consistent across restarts."],"tags":["kinesis","partition-ownership","internal-invariant","rust"],"backgroundTag":"internal-invariant-violation","analyzedSha":"a39730c5cdcd1a4fe798403737ae293999ea21f8","analyzedAt":"2026-09-08T13:19:37.784Z","contentChangedAt":"2026-09-08T13:19:37.784Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}