quickwit-oss/quickwit · error

`doc_batch` should not be empty

Error message

`doc_batch` should not be empty

What it means

to_simple_list is a serde serializer helper for `Option<Vec<T>>` fields. It calls `.as_ref().expect(...)` on the Option, so serializing a field whose value is None panics. The helper assumes callers only ever attach it to fields already guaranteed to be Some (e.g. via `#[serde(serialize_with = "to_simple_list")]` on an always-populated field).

Source

Thrown at quickwit/quickwit-ingest/src/ingest_v2/mrecordlog_utils.rs:90

            .await
    } else {
        let encoded_mrecords = doc_batch
            .into_docs()
            .map(|(_doc_uid, doc)| MRecord::Doc(doc).encode());

        #[cfg(feature = "failpoints")]
        fail_point!("ingester:append_records", |_| {
            let io_error = io::Error::from(io::ErrorKind::PermissionDenied);
            Err(AppendDocBatchError::Io(io_error))
        });

        mrecordlog
            .append_records(queue_id, None, encoded_mrecords)
            .await
    };
    match append_result {
        Ok(Some(offset)) => Ok(Position::offset(offset)),
        Ok(None) => panic!("`doc_batch` should not be empty"),
        Err(AppendError::IoError(io_error)) => Err(AppendDocBatchError::Io(io_error)),
        Err(AppendError::MissingQueue(queue_id)) => {
            Err(AppendDocBatchError::QueueNotFound(queue_id))
        }
        Err(AppendError::Past) => {
            panic!("`append_records` should be called with `position_opt: None`")
        }
    }
}

/// Error returned when the mrecordlog does not have enough capacity to store some records.
#[derive(Debug, Clone, Copy, thiserror::Error)]
pub(super) enum NotEnoughCapacityError {
    #[error(
        "write-ahead log is full, capacity: {capacity}, usage: {usage}, requested: {requested}"
    )]
    Disk {
        usage: ByteSize,

View on GitHub (pinned to a39730c5cd)

Solutions

1. Ensure the field is Some before serialization — populate the list explicitly when constructing the struct.
2. Change the field type to Vec<T> with a default (`#[serde(default)]`) so it can never be None.
3. Add a custom serializer that renders None as an empty string instead of panicking.
4. Fix the source data (config/index metadata file) so the field is present and non-null.

Example fix

// before
let vec = &value
    .as_ref()
    .expect("attempt to serialize Option::None value");
// after
let Some(vec) = value.as_ref() else {
    serializer.serialize_str("")?; // or return serde::private::de::missing_field error
    return Ok(());
};

// in Rust, before serializing a struct using to_simple_list fields:
assert!(some_struct.retention_period_opt.is_some(), "list field must be Some before serialization");

fn ensure_some_list<T>(opt: &Option<Vec<T>>) -> Option<&Vec<T>> { opt.as_ref() }

// guard deserialized structs before re-serialization
if struct_with_optional_list.list_field.is_none() {
    return Err(anyhow!("missing list field in metadata; cannot serialize"));
}

When it happens

Trigger: Serializing (e.g. via serde_json) a struct whose field uses `serialize_with = "to_simple_list"` while that field holds None — typically constructing the struct with None for a list field that the helper was designed for non-None values.

Common situations: Deserializing a config/metadata file where the list field is absent or null and then re-serializing it; programmatically building index metadata/structs with None where a list is required; older metadata files written by a previous version lacking the field.

Related errors

• serializing PartialHit should never fail
• QueryAst should be JSON serializable
• Failed to serialize QueryAst
• node not found in pending
• OTP logs or traces do not support VRL transforms

AI-assisted analysis of quickwit-oss/quickwit@a39730c5cd (2026-09-08). Data as JSON: /api/errors/9ca0c63ff1304b4c. Report an issue: GitHub.