{"record":{"id":"67fe620e4e64b702","repo":"nautechsystems/nautilus_trader","slug":"e-67fe62","errorCode":null,"errorMessage":"{e}","messagePattern":"\\{e\\}","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"crates/serialization/src/arrow/custom.rs","lineNumber":115,"sourceCode":"\n    let schema = Arc::new(T::get_schema(None));\n\n    let encoder: ArrowEncoder = Box::new(|items: &[Arc<dyn CustomDataTrait>]| {\n        let typed: Result<Vec<T>, _> = items\n            .iter()\n            .map(|b| {\n                b.as_any()\n                    .downcast_ref::<T>()\n                    .cloned()\n                    .ok_or_else(|| anyhow::anyhow!(\"Expected {}\", T::type_name_static()))\n            })\n            .collect();\n        let typed = typed?;\n        let metadata = typed\n            .first()\n            .map(EncodeToRecordBatch::metadata)\n            .unwrap_or_default();\n        EncodeToRecordBatch::encode_batch(&metadata, &typed).map_err(|e| anyhow::anyhow!(\"{e}\"))\n    });\n\n    let decoder: ArrowDecoder = Box::new(|metadata, batch| {\n        T::decode_data_batch(metadata, batch).map_err(|e| anyhow::anyhow!(\"{e}\"))\n    });\n\n    let _ = ensure_arrow_registered(type_name, schema, encoder, decoder);\n}\n\n/// Decoder for custom data types that are identified at runtime by metadata (e.g. `type_name`).\n///\n/// Only Rust-registered custom types (e.g. `RustTestCustomData`, `MacroYieldCurveData`) can be\n/// decoded. Unknown types return an error.\n///\n/// **Important:** The caller must ensure that any Rust custom data types are registered\n/// via [`ensure_custom_data_registered::<T>()`] before use.\n#[derive(Debug)]\npub struct CustomDataDecoder;","sourceCodeStart":97,"sourceCodeEnd":133,"githubUrl":"https://github.com/nautechsystems/nautilus_trader/blob/18893faf8b356be3320add8de2f861b0b647cf06/crates/serialization/src/arrow/custom.rs#L97-L133","documentation":"In ensure_custom_data_registered, after downcasting arrays to the concrete type, the encoder calls EncodeToRecordBatch::encode_batch; any error returned by the type's own encode implementation is wrapped verbatim into an anyhow error with this message. The underlying cause is whatever the custom type's encode_batch reported (e.g. mismatched metadata, wrong number of columns).","triggerScenarios":"Encoding custom data where T::encode_batch fails — typically metadata mismatch between the declared EncodeToRecordBatch metadata and the typed items, or a field that cannot be converted to its Arrow representation.","commonSituations":"Custom data types with hand-written EncodeToRecordBatch impls that drifted from the struct fields; empty or mismatched metadata dictionaries; nulls in fields the encoder assumes non-null.","solutions":["Read the wrapped inner error ({e}) for the concrete cause — it names the failing encode step","Verify the custom type's metadata() output matches the data being encoded (field count, types, ordering)","Fix the type's EncodeToRecordBatch implementation if struct fields changed without updating the encoder","Add a unit test encoding a representative instance of the custom type to catch drift early"],"exampleFix":"// before: metadata defined manually, drifted from fields\nfn metadata() -> Metadata { /* stale fields */ }\n// after: derive metadata from the struct definition so it cannot drift\nfn metadata() -> Metadata {\n    Metadata::from_fields(&[(\"price\", ArrowType::Float64), (\"qty\", ArrowType::UInt64)])\n}","handlingStrategy":"try-catch","validationCode":"let items_typed: Vec<MyData> = ...;\nlet metadata = <MyData as EncodeToRecordBatch>::metadata();\nassert_eq!(items_typed.len(), expected_len, \"encoder input must be non-empty and consistent\");","typeGuard":null,"tryCatchPattern":"match ensure_custom_data_registered::<MyData>() {\n    Err(e) => {\n        // {e} wraps the encode_batch error verbatim — log with the full chain\n        log::error!(\"custom data encode failed: {e:#}\");\n    }\n    Ok(()) => (),\n}","preventionTips":["Derive metadata from the struct definition instead of hand-maintaining it","Round-trip test (encode then decode) every custom data type in CI","Handle nullability explicitly in encode implementations"],"tags":["arrow","serialization","encoding","custom-data"],"backgroundTag":"json-serialization-failed","analyzedSha":"18893faf8b356be3320add8de2f861b0b647cf06","analyzedAt":"2026-09-08T20:49:34.690Z","contentChangedAt":"2026-09-08T20:49:34.690Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}