nautechsystems/nautilus_trader · error

Expected {}

Error message

Expected {}

What it means

ensure_custom_data_registered builds an Arrow encoder for a custom data type T by downcasting each Arrow array in the input to the concrete array type T expects. If any element in the batch cannot be downcast to T's expected array type (ArrayRef -> T's array), the registration fails with "Expected {type_name}". This is an internal invariant: the batch passed to the encoder must match the registered type's schema.

Source

Thrown at crates/serialization/src/arrow/custom.rs:107

    let type_name = T::type_name_static();

    // Skip if already registered
    if get_arrow_schema(type_name).is_some() {
        return;
    }

    let _ = ensure_custom_data_json_registered::<T>();

    let schema = Arc::new(T::get_schema(None));

    let encoder: ArrowEncoder = Box::new(|items: &[Arc<dyn CustomDataTrait>]| {
        let typed: Result<Vec<T>, _> = items
            .iter()
            .map(|b| {
                b.as_any()
                    .downcast_ref::<T>()
                    .cloned()
                    .ok_or_else(|| anyhow::anyhow!("Expected {}", T::type_name_static()))
            })
            .collect();
        let typed = typed?;
        let metadata = typed
            .first()
            .map(EncodeToRecordBatch::metadata)
            .unwrap_or_default();
        EncodeToRecordBatch::encode_batch(&metadata, &typed).map_err(|e| anyhow::anyhow!("{e}"))
    });

    let decoder: ArrowDecoder = Box::new(|metadata, batch| {
        T::decode_data_batch(metadata, batch).map_err(|e| anyhow::anyhow!("{e}"))
    });

    let _ = ensure_arrow_registered(type_name, schema, encoder, decoder);
}

/// Decoder for custom data types that are identified at runtime by metadata (e.g. `type_name`).

View on GitHub (pinned to 18893faf8b)

Solutions

1. Ensure the RecordBatch passed to the encoder was produced by the same T's encode implementation (same type_name and schema)
2. Check for schema drift: if the custom data struct's fields/types changed, re-encode historical data or bump the type/schema identifier
3. Validate array types before encoding (e.g. assert batch columns downcast to the expected Arrow array types)
4. Regenerate or update any persisted Arrow files written with the old schema

Example fix

// before: passing a generic batch to the custom encoder
ensure_custom_data_registered::<MyData>(&batches)?;
// after: filter batches to those produced for MyData
let my_batches: Vec<_> = batches.into_iter().filter(|b| b.schema() == MyData::schema()).collect();
ensure_custom_data_registered::<MyData>(&my_batches)?;

fn batch_matches<T: EncodeToRecordBatch>(batch: &RecordBatch) -> bool {
    batch.columns().iter().all(|c| c.as_any().downcast_ref::<T::Array>().is_some())
}

fn as_expected_array<T: 'static>(arr: &dyn Array) -> Option<&T> {
    arr.as_any().downcast_ref::<T>()
}

match ensure_custom_data_registered::<MyData>(...) {
    Err(e) if e.to_string().starts_with("Expected ") => {
        eprintln!("schema mismatch in custom data batch: {e:#}");
    }
    other => other?,
}

When it happens

Trigger: Registering or encoding custom data where the RecordBatch contains arrays of a different concrete Arrow type than T::EncodeToRecordBatch produces — e.g. schema drift between the writer and the registered encoder, or mixed array types in one batch.

Common situations: Custom data structs whose encode implementation changed (field type changed from Int64 to Float64) while old files/streams still use the old schema; passing a batch produced for a different data type into the encoder; version mismatch between serialization code writing the data and code reading it.

Understand the failure class

Background: Type mismatch errors: IllegalArgumentException, TypeError and type guards across 150 open-source libraries — this error's family across 150 libraries.

Related errors

• height must be positive, was {self.height}
• pandas is required for report generation; install it with `p
• pandas is required for visualization; install it with `pip i
• {name} must not be None
• Chart name cannot be empty

AI-assisted analysis of nautechsystems/nautilus_trader@18893faf8b (2026-09-08). Data as JSON: /api/errors/1b485a33b3687e08. Report an issue: GitHub.