quickwit-oss/quickwit · error

failed to find offset for split {}

Error message

failed to find offset for split {}

What it means

During the fetch-docs phase of distributed search, results grouped by split_id are looked up in a map of split_id -> split offsets built from the search hits. If a split appears in the hit groups but has no entry in the split offsets map, this invariant is violated and the error is thrown. It indicates the caller-supplied hit list references a split that was never resolved to offsets.

Source

Thrown at quickwit/quickwit-search/src/fetch_docs.rs:67

    let mut split_fetch_docs_futures = Vec::new();

    let split_offsets_map: HashMap<&str, &SplitIdAndFooterOffsets> = splits
        .iter()
        .map(|split| (split.split_id.as_str(), split))
        .collect();

    // We sort global hit addrs in order to allow for the grouby.
    global_doc_addrs.sort_by(|a, b| a.split.cmp(&b.split));
    for (split_id, global_doc_addrs) in global_doc_addrs
        .iter()
        .chunk_by(|global_doc_addr| global_doc_addr.split.as_str())
        .into_iter()
    {
        let global_doc_addrs: Vec<GlobalDocAddress> =
            global_doc_addrs.into_iter().cloned().collect();
        let split_and_offset = split_offsets_map
            .get(split_id)
            .ok_or_else(|| anyhow::anyhow!("failed to find offset for split {}", split_id))?;
        let split_id = split_id.to_string();
        split_fetch_docs_futures.push(
            fetch_docs_in_split(
                searcher_context.clone(),
                global_doc_addrs,
                index_storage.clone(),
                split_and_offset,
                doc_mapper.clone(),
                snippet_request_opt,
            )
            .map_err(move |e| e.context(format!("split_id={split_id}"))),
        );
    }

    let split_fetch_docs: Vec<Vec<(GlobalDocAddress, Document)>> =
        futures::future::try_join_all(split_fetch_docs_futures)
            .await
            .map_err(|error| {

View on GitHub (pinned to a39730c5cd)

Solutions

1. Verify the split_id in the error actually exists in the index and its splits were resolved before fetch_docs (check the search response's split_offsets).
2. Re-run the search against a fresh metastore snapshot; the split may have been removed mid-flight.
3. If building hits programmatically, ensure every hit's split is passed through the same offset-resolution step that populates split_offsets_map.

let missing: Vec<_> = hits.iter().map(|h| h.split_id()).filter(|s| !split_offsets_map.contains_key(*s)).collect();
if !missing.is_empty() { return Err(anyhow!("unresolved splits: {missing:?}")); }

match fetch_docs(...).await {
    Ok(docs) => docs,
    Err(e) if e.to_string().contains("failed to find offset for split") => {
        // stale search response; retry the search to rebuild split offsets
        retry_search().await
    }
    Err(e) => return Err(e.into()),
}

When it happens

Trigger: Calling fetch_docs with a set of hits whose split_id has no corresponding entry in split_offsets_map — e.g. hits from a metastore/search response that include a split not covered by the offset resolution step.

Common situations: Stale or mismatched search responses (split deleted between search and fetch); custom search callers constructing hits manually; internal inconsistencies after split migration or retention deletion.

Understand the failure class

Background: Record Not Found Errors: "not found", RecordNotFound, and "was not found" — what they mean and how to fix them — this error's family across 28 libraries.

Related errors

• unknown tokenizer `{}` for field `{}`
• invalid named document. there are more than 1 value associat
• the `{key}` value has to be a json object
• failed to assign search jobs: there are no available searche
• failed to assign search jobs: there are no searcher nodes ca

AI-assisted analysis of quickwit-oss/quickwit@a39730c5cd (2026-09-08). Data as JSON: /api/errors/bd5780963f4bf5f1. Report an issue: GitHub.