quickwit-oss/quickwit · error

`index_uid` should be a required field

Error message

`index_uid` should be a required field

What it means

When building the Elasticsearch-compatible bulk response v2, each IngestSuccess subresponse must carry an index_uid so the code can map results back to per-request doc handles. The gRPC contract marks index_uid as required, so a None here means the ingest service violated the protocol, and the code panics via expect.

Source

Thrown at quickwit/quickwit-serve/src/elasticsearch_api/bulk_v2.rs:186

#[allow(clippy::result_large_err)]
fn make_elastic_bulk_response_v2(
    ingest_response_v2: IngestResponseV2,
    mut per_subrequest_doc_handles: HashMap<u32, Vec<DocHandle>>,
    now: Instant,
    action_count: usize,
    invalid_index_id_items: Vec<(usize, ElasticBulkItem)>,
) -> Result<ElasticBulkResponse, ElasticsearchError> {
    let mut positioned_actions: Vec<(usize, ElasticBulkAction)> = Vec::with_capacity(action_count);
    let mut errors = false;

    // Populate the items for each `IngestSuccess` subresponse. They may be partially successful and
    // contain some parse failures.
    for success in ingest_response_v2.successes {
        let index_id = success
            .index_uid
            .map(|index_uid| index_uid.index_id)
            .expect("`index_uid` should be a required field");

        // Find the doc handles for the subresponse.
        let mut doc_handles = remove_doc_handles(
            &mut per_subrequest_doc_handles,
            success.subrequest_id,
        )
        .inspect_err(|_| {
            rate_limited_error!(limit_per_min=6, index_id=%index_id, "could not find subrequest id");
        })?;
        doc_handles.sort_unstable_by_key(|doc_handle| doc_handle.doc_uid);

        // Populate the response items with one error per parse failure.
        for parse_failure in success.parse_failures {
            errors = true;

            let failed_doc_uid = parse_failure.doc_uid();
            let doc_handle_idx = doc_handles
                .binary_search_by_key(&failed_doc_uid, |doc_handle| doc_handle.doc_uid)

View on GitHub (pinned to a39730c5cd)

Solutions

1. Ensure the ingest API service and serve crate versions match
2. Check that the ingest service populates index_uid on every IngestSuccess
3. If a custom/mock service is used, set index_uid in all success entries
4. Consider converting this to a returned error instead of a panic for robustness

Example fix

// before
let index_id = success.index_uid.map(|u| u.index_id).expect("`index_uid` should be a required field");
// after
let index_id = success
    .index_uid
    .map(|u| u.index_id)
    .ok_or_else(|| ElasticsearchError::from(anyhow!("ingest success missing index_uid")))?;

// Validate each success entry before consuming:
for success in &resp.successes {
    if success.index_uid.is_none() { return Err(anyhow!("ingest success missing index_uid")); }
}

fn has_index_uid(success: &IngestSuccess) -> bool {
    success.index_uid.is_some()
}

// If calling the v2 ingest API through a custom client, wrap:
let resp = client.ingest_v2(req).await?;
if resp.successes.iter().any(|s| s.index_uid.is_none()) {
    return Err(anyhow!("malformed v2 ingest response: missing index_uid"));
}

When it happens

Trigger: elastic_bulk_ingest_v2 receives a v2 ingest response whose successes contain an entry with no index_uid set (ingest service bug or cross-version protocol mismatch).

Common situations: Running a quickwit server with mismatched grpc service versions (older ingest service producing v2 responses without index_uid); custom/mocked ingest services in tests returning incomplete successes.

Related errors

• tasks running the gRPC server should not panic or be cancell
• node not found in pending
• OTP logs or traces do not support VRL transforms
• position of a Kafka partition should never be EOF
• position of a Kinesis shard should never be EOF

AI-assisted analysis of quickwit-oss/quickwit@a39730c5cd (2026-09-08). Data as JSON: /api/errors/ebdc1b95eec550ab. Report an issue: GitHub.