nautechsystems/nautilus_trader · error

Chunk should have at least one element to encode

Error message

Chunk should have at least one element to encode

What it means

chunk_metadata is part of the Arrow encoding path for Databento records: it inspects the first element of a chunk to derive metadata (price/size precision) for the record batch. The expect asserts the chunk is non-empty; encoding an empty slice is unsupported because there is no record to derive precision metadata from, so it panics instead of producing an empty batch with metadata.

Solutions

1. Guard the caller: skip encoding (or emit an empty RecordBatch without metadata) when the chunk is empty.
2. Fix chunk-boundary computation so empty chunks are never produced.
3. Ensure the data source query/subscription actually yields rows before invoking the encoder.

Example fix

// before
encode(chunk); // panics when chunk.is_empty()
// after
if !chunk.is_empty() {
    encode(chunk);
} else {
    // skip or emit empty batch without metadata
}

if chunk.is_empty() {
    return Ok(()); // or emit an empty batch without metadata
}
encode_chunk(chunk);

// Avoid panics upstream by validating before the encoder; catch only as a safety net:
std::panic::catch_unwind(|| encode_chunk(chunk)).map_err(|_| anyhow!("empty chunk encoded"))

When it happens

Trigger: Calling the encode/chunking API with an empty slice of statistics records (chunk.len() == 0), typically after filtering a batch to nothing or miscomputing chunk boundaries so the last chunk is empty.

Common situations: Upstream data source returned no rows for a subscription window; a filtering step removed all records; off-by-one chunking produces trailing empty chunks fed to the encoder.

Understand the failure class

Background: "must not be empty", "cannot be empty" — required-field validation errors across open-source libraries — this error's family across 41 libraries.

Related errors

• 16 lowercase hex bytes are valid TradeId
• API key is valid UTF-8
• bundled America/New_York timezone
• No symbols provided
• OrderFactory::create_list requires non-empty orders

AI-assisted analysis of nautechsystems/nautilus_trader@18893faf8b (2026-09-08). Data as JSON: /api/errors/2b39e636869a6a3c. Report an issue: GitHub.

Appendix: source

Thrown at crates/adapters/databento/src/arrow/statistics.rs:143

                Arc::new(ts_event_builder.finish()),
                Arc::new(ts_recv_builder.finish()),
                Arc::new(ts_init_builder.finish()),
            ],
        )
    }

    fn metadata(&self) -> HashMap<String, String> {
        Self::get_metadata(
            &self.instrument_id,
            self.price.map_or(FIXED_PRECISION, |p| p.precision),
            self.quantity.map_or(FIXED_PRECISION, |q| q.precision),
        )
    }

    fn chunk_metadata(chunk: &[Self]) -> HashMap<String, String> {
        let first = chunk
            .first()
            .expect("Chunk should have at least one element to encode");

        let price_precision = chunk
            .iter()
            .find_map(|s| s.price.map(|p| p.precision))
            .unwrap_or(FIXED_PRECISION);
        let size_precision = chunk
            .iter()
            .find_map(|s| s.quantity.map(|q| q.precision))
            .unwrap_or(FIXED_PRECISION);

        Self::get_metadata(&first.instrument_id, price_precision, size_precision)
    }
}

impl DecodeDataFromRecordBatch for DatabentoStatistics {
    fn decode_data_batch(
        metadata: &HashMap<String, String>,
        record_batch: RecordBatch,

View on GitHub (pinned to 18893faf8b)