nautechsystems/nautilus_trader · error · anyhow::Error

Expected {}, was different type

Error message

Expected {}, was different type

What it means

The #[custom_data] macro generates a serialize/encode implementation that downcasts each incoming Data trait object to the concrete type Self. If any item is not actually an instance of the macro-generated type, generation bails with 'Expected {TypeName}, was different type', protecting the batch encoder from mixing incompatible payloads.

Source

Thrown at crates/persistence/macros/src/custom.rs:966

    let generics = ctx.generics;
    let name_str = ctx.name_str;
    quote! {
        impl #generics nautilus_serialization::arrow::custom::CustomDataSerialize for #name #generics {
            fn schema(&self) -> anyhow::Result<arrow::datatypes::Schema> {
                Ok(<Self as nautilus_serialization::arrow::ArrowSchemaProvider>::get_schema(
                    Some(nautilus_serialization::arrow::EncodeToRecordBatch::metadata(self))
                ).into())
            }
            fn encode_record_batch(
                &self,
                items: &[std::sync::Arc<dyn nautilus_model::data::CustomDataTrait>],
            ) -> anyhow::Result<arrow::record_batch::RecordBatch> {
                let mut typed: Vec<Self> = Vec::with_capacity(items.len());
                for item in items {
                    if let Some(c) = item.as_any().downcast_ref::<Self>() {
                        typed.push(std::clone::Clone::clone(c));
                    } else {
                        anyhow::bail!("Expected {}, was different type", #name_str);
                    }
                }
                let metadata = nautilus_serialization::arrow::EncodeToRecordBatch::metadata(self);
                nautilus_serialization::arrow::EncodeToRecordBatch::encode_batch(&metadata, &typed).map_err(Into::into)
            }
        }
    }
}

fn gen_arrow_schema_impl(ctx: &ExpansionContext<'_>) -> TokenStream {
    let name = ctx.name;
    let generics = ctx.generics;
    let field_list = ctx.field_list;
    let arrow_schema_fields: Vec<TokenStream> = field_list
        .iter()
        .map(|f| {
            let ident = &f.ident;
            let ty = &f.ty;

View on GitHub (pinned to 18893faf8b)

Solutions

1. Ensure the batch passed to the generated encode function contains only instances of that exact custom data type.
2. Separate publishing/encoding paths per data type instead of funneling all Data through one encoder.
3. Check the downcast at the producer side: filter or split items by concrete type before batching.
4. Verify custom data registration maps each type to its own encoder metadata.

Example fix

// before
let batch = my_data_encode(all_items); // all_items mixes Data types
// after
let typed: Vec<MyData> = all_items.into_iter().filter_map(|d| d.as_any().downcast_ref::<MyData>().cloned()).collect();
let batch = my_data_encode(typed);

// Rust: filter to the concrete type before encoding
let typed: Vec<MyData> = items.into_iter()
    .filter_map(|d| d.as_any().downcast_ref::<MyData>().cloned())
    .collect();

fn is_my_data(d: &dyn Data) -> bool { d.as_any().downcast_ref::<MyData>().is_some() }

match encoder.encode_batch(items) {
    Ok(batch) => batch,
    Err(e) => { log::error!("encode failed (mixed types?): {e}"); split_by_type_and_retry(items) },
}

When it happens

Trigger: expand_custom_data / gen_custom_data_serialize_impl emitting code that is later called with a Vec of Data objects containing items of another custom data type — e.g. encoding a batch that mixes MyData and OtherData, or passing the wrong subscription's messages to the encoder.

Common situations: Publishing two different custom data types through one queue/topic and then encoding them together; generic pipeline code assuming a homogeneous Vec<Data>; registration wiring that associates the wrong data type with an encoder.

Understand the failure class

Background: Type mismatch errors: IllegalArgumentException, TypeError and type guards across 150 open-source libraries — this error's family across 150 libraries.

Related errors

• Instances must have encode_record_batch_py method
• Expected {}
• Expected Custom data variant
• ts_init column is not UInt64
• Failed to encode custom data to Arrow: {e}

AI-assisted analysis of nautechsystems/nautilus_trader@18893faf8b (2026-09-08). Data as JSON: /api/errors/b5f8246995a115c7. Report an issue: GitHub.