nautechsystems/nautilus_trader · error

Failed to call from_json: {e}

Error message

Failed to call from_json: {e}

What it means

After parsing the JSON dict, py_json_deserialize_custom_data calls the registered Python class's from_json(py_dict) classmethod to build the instance. If that Python method raises or does not exist, the error is wrapped as 'Failed to call from_json: {e}'. This means the custom data class's Python-side contract is broken.

Source

Thrown at crates/model/src/python/data/mod.rs:365

    value: &serde_json::Value,
) -> Result<std::sync::Arc<dyn crate::data::CustomDataTrait>, anyhow::Error> {
    use std::sync::Arc;

    use crate::data::PythonCustomDataWrapper;

    pyo3::Python::attach(|py| {
        let json_str = serde_json::to_string(&value)?;
        let json_module = py
            .import("json")
            .map_err(|e| anyhow::anyhow!("Failed to import json: {e}"))?;
        let py_dict = json_module
            .call_method1("loads", (json_str,))
            .map_err(|e| anyhow::anyhow!("Failed to parse JSON: {e}"))?;

        let instance = data_class
            .bind(py)
            .call_method1("from_json", (py_dict,))
            .map_err(|e| anyhow::anyhow!("Failed to call from_json: {e}"))?;

        let wrapper = PythonCustomDataWrapper::new(py, &instance)
            .map_err(|e| anyhow::anyhow!("Failed to create wrapper: {e}"))?;

        Ok(Arc::new(wrapper) as Arc<dyn crate::data::CustomDataTrait>)
    })
}

/// Encodes `CustomData` items to `RecordBatch` via Python `encode_record_batch_py`.
#[allow(unsafe_code)]
#[cfg(all(feature = "python", feature = "arrow"))]
fn py_encode_custom_data_to_record_batch(
    items: &[std::sync::Arc<dyn crate::data::CustomDataTrait>],
) -> Result<arrow::record_batch::RecordBatch, anyhow::Error> {
    pyo3::Python::attach(|py| {
        let py_items: Result<Vec<_>, _> = items.iter().map(|item| item.to_pyobject(py)).collect();
        let py_items = py_items.map_err(|e| anyhow::anyhow!("Failed to convert to Python: {e}"))?;
        let py_list = pyo3::types::PyList::new(py, &py_items)

View on GitHub (pinned to 18893faf8b)

Solutions

  1. Implement (or fix) from_json on the registered Python class as a classmethod accepting the dict produced by serialization.
  2. Make from_json tolerant of missing/extra keys matching the Rust struct's serde field names (check rename_all attributes).
  3. Test the class directly: MyClass.from_json(json.loads(MyClass(...).to_json())) in Python before registering.
  4. Check for schema drift between the Rust struct and the Python class after upgrades and keep field names in sync.

Example fix

// before: class lacks from_json
#[pyclass]
struct MyData { /* ... */ }
// after: add the contract on the Python side
class MyData:
    @classmethod
    def from_json(cls, data: dict) -> "MyData":
        return cls(
            value=data["value"],           # keys must match serde field names
            ts_event=data["ts_event"],
            ts_init=data["ts_init"],
        )
Defensive patterns

Strategy: validation

Validate before calling

# verify before registering the class
import inspect, json
assert hasattr(MyData, "from_json"), "custom data class must define from_json classmethod"
sig = inspect.signature(MyData.from_json)
assert "data" in sig.parameters, "from_json must accept the deserialized dict"

Try / catch

match py_json_deserialize_custom_data(data_class, value) {
    Ok(v) => v,
    Err(e) if e.to_string().contains("Failed to call from_json") => {
        log::error!("custom data class from_json broken: {e}; check field names vs serde schema");
        Err(e)
    }
    Err(e) => return Err(e),
}

Prevention

When it happens

Trigger: register_custom_data_class was given a Python class that has no from_json classmethod, from_json has an incompatible signature, or from_json raises (KeyError on missing fields, TypeError on wrong types, validation errors) when given the deserialized dict.

Common situations: Custom data class written without the required from_json classmethod; Rust struct fields renamed/added so the Python from_json no longer matches the JSON keys; from_json expecting a JSON string but receiving a dict after a version change.

Related errors

AI-assisted analysis of nautechsystems/nautilus_trader@18893faf8b (2026-09-08). Data as JSON: /api/errors/94fdc9bf37c31484. Report an issue: GitHub.