nautechsystems/nautilus_trader · error

Failed to create list: {e}

Error message

Failed to create list: {e}

What it means

After converting custom data items to Python objects, the encoder builds a `PyList` of them to pass to the Python-side `encode_record_batch_py`. Pyo3's `PyList::new` returns an error if any element cannot be converted to a Python object or the list construction fails; that error is wrapped as "Failed to create list". This is a preparatory step before calling into Python, so the whole encode fails.

Source

Thrown at crates/model/src/python/data/mod.rs:384

        let wrapper = PythonCustomDataWrapper::new(py, &instance)
            .map_err(|e| anyhow::anyhow!("Failed to create wrapper: {e}"))?;

        Ok(Arc::new(wrapper) as Arc<dyn crate::data::CustomDataTrait>)
    })
}

/// Encodes `CustomData` items to `RecordBatch` via Python `encode_record_batch_py`.
#[allow(unsafe_code)]
#[cfg(all(feature = "python", feature = "arrow"))]
fn py_encode_custom_data_to_record_batch(
    items: &[std::sync::Arc<dyn crate::data::CustomDataTrait>],
) -> Result<arrow::record_batch::RecordBatch, anyhow::Error> {
    pyo3::Python::attach(|py| {
        let py_items: Result<Vec<_>, _> = items.iter().map(|item| item.to_pyobject(py)).collect();
        let py_items = py_items.map_err(|e| anyhow::anyhow!("Failed to convert to Python: {e}"))?;
        let py_list = pyo3::types::PyList::new(py, &py_items)
            .map_err(|e| anyhow::anyhow!("Failed to create list: {e}"))?;

        let first = items
            .first()
            .ok_or_else(|| anyhow::anyhow!("No items to encode"))?;
        let first_py = first.to_pyobject(py)?;

        if first_py
            .bind(py)
            .hasattr("encode_record_batch_py")
            .unwrap_or(false)
        {
            let py_batch = first_py
                .bind(py)
                .call_method1("encode_record_batch_py", (py_list,))
                .map_err(|e| anyhow::anyhow!("Failed to call encode_record_batch_py: {e}"))?;

            let mut ffi_array = arrow::ffi::FFI_ArrowArray::empty();
            let mut ffi_schema = arrow::ffi::FFI_ArrowSchema::empty();

View on GitHub (pinned to 18893faf8b)

Solutions

  1. Check the embedded `{e}` for the underlying pyo3 conversion error.
  2. Confirm the items were valid Python objects (the preceding 'Failed to convert to Python' step succeeded).
  3. Upgrade or align pyo3 versions between nautilus crates and any custom build code.
  4. Retry the encode in a fresh interpreter/GIL context if running embedded.
Defensive patterns

Strategy: try-catch

Try / catch

try:
    batch = encode_to_record_batch(items)
except Exception as e:
    logger.error(f"PyList construction failed: {e}")
    raise

Prevention

When it happens

Trigger: Calling the Arrow encode path for custom data when `pyo3::types::PyList::new(py, &py_items)` fails — rare in practice since items were already converted, but possible if an element panics or conversion to a Python object fails during list materialization.

Common situations: Pyo3 version edge cases with `IntoPyObject`/`ToPyObject` conversions for wrapper types; embedding scenarios where the Python interpreter is in a bad state; items containing objects that fail conversion when collected into the list.

Understand the failure class

Background: "JSON serialization failed", "not JSON serializable", "Failed to serialize": why JSON marshaling errors happen and how to fix them — this error's family across 46 libraries.

Related errors

AI-assisted analysis of nautechsystems/nautilus_trader@18893faf8b (2026-09-08). Data as JSON: /api/errors/fe488cc9a3e08f66. Report an issue: GitHub.