nautechsystems/nautilus_trader · error · anyhow::Error

decode_custom_batches_to_data called with empty batches

Error message

decode_custom_batches_to_data called with empty batches

What it means

Thrown by decode_custom_batches_to_data when it is called with an empty iterator of RecordBatches. The function derives the schema from the first batch, so with zero batches there is nothing to decode and no schema to consult; the call is treated as a programming error rather than returning an empty result.

Source

Thrown at crates/persistence/src/backend/custom.rs:243

    }
}

/// Decodes multiple `RecordBatches` (e.g. from custom data files) into a single `Vec<Data>`.
/// Optionally replaces `ts_init` column with `ts_event` before decoding each batch.
///
/// # Errors
///
/// Returns an error if any batch fails to decode.
pub fn decode_custom_batches_to_data(
    batches: Vec<RecordBatch>,
    use_ts_event_for_ts_init: bool,
) -> anyhow::Result<Vec<Data>> {
    let mut file_data = Vec::new();
    let schema = batches
        .first()
        .map(arrow::array::RecordBatch::schema)
        .ok_or_else(|| {
            anyhow::anyhow!("decode_custom_batches_to_data called with empty batches")
        })?;

    for mut batch in batches {
        if use_ts_event_for_ts_init {
            let column_names: Vec<String> =
                schema.fields().iter().map(|f| f.name().clone()).collect();

            if let (Some(ts_event_idx), Some(ts_init_idx)) = (
                column_names.iter().position(|n| n == "ts_event"),
                column_names.iter().position(|n| n == "ts_init"),
            ) {
                let mut new_columns = batch.columns().to_vec();
                new_columns[ts_init_idx] = new_columns[ts_event_idx].clone();
                batch = RecordBatch::try_new(schema.clone(), new_columns)
                    .map_err(|e| anyhow::anyhow!("Failed to create new batch: {e}"))?;
            }
        }
        let metadata = batch.schema().metadata().clone();

View on GitHub (pinned to 18893faf8b)

Solutions

1. Guard the caller: check !batches.is_empty() before calling decode_custom_batches_to_data and return an empty Vec<Data> instead.
2. Fix the upstream query/write logic if empty files are being produced unexpectedly.
3. If an empty result is legitimate, treat it as no-op at the call site rather than invoking the decoder.

Example fix

// before
let data = decode_custom_batches_to_data(&batches, false, true)?;
// after
let data = if batches.is_empty() {
    Vec::new()
} else {
    decode_custom_batches_to_data(&batches, false, true)?
};

if batches.is_empty() { return Ok(Vec::new()); }

fn non_empty(batches: &[RecordBatch]) -> Option<&[RecordBatch]> {
    (!batches.is_empty()).then_some(batches)
}

match decode_custom_batches_to_data(&batches, false, true) {
    Ok(data) => use(data),
    Err(e) if e.to_string().contains("empty batches") => return Ok(Vec::new()),
    Err(e) => return Err(e),
}

When it happens

Trigger: Calling decode_custom_batches_to_data with an empty Vec of batches — e.g. a file query returned no record batches, or write/read logic filtered out all batches before the call.

Common situations: Reading a catalog file that contained zero rows after a filtered query; code paths that pass batches.first()-derived collections without checking emptiness; handling of empty feather files produced by failed writes.

Understand the failure class

Background: "must not be empty", "cannot be empty" — required-field validation errors across open-source libraries — this error's family across 41 libraries.

Related errors

• prepare_custom_data_batch called with empty data
• No account events provided to create `AccountAny`
• Expected {}
• Expected Custom data variant
• Unsupported Data::Defi variant for catalog writes

AI-assisted analysis of nautechsystems/nautilus_trader@18893faf8b (2026-09-08). Data as JSON: /api/errors/afa231e823c97e0d. Report an issue: GitHub.