{"record":{"id":"ea0b57252aa4decc","repo":"nautechsystems/nautilus_trader","slug":"missing-type-name-in-metadata","errorCode":null,"errorMessage":"Missing type_name in metadata","messagePattern":"Missing type_name in metadata","errorType":"exception","errorClass":"anyhow::Error","httpStatus":null,"severity":"error","filePath":"crates/persistence/src/backend/custom.rs","lineNumber":188,"sourceCode":"/// produce an error instead of attempting custom decode.\n///\n/// # Errors\n///\n/// Returns an error if decoding fails or the type is unknown (and custom fallback not allowed).\n#[expect(\n    clippy::implicit_hasher,\n    reason = \"Arrow schema metadata uses the standard HashMap type\"\n)]\npub fn decode_batch_to_data(\n    metadata: &HashMap<String, String>,\n    batch: RecordBatch,\n    allow_custom_fallback: bool,\n) -> anyhow::Result<Vec<Data>> {\n    let type_name = metadata\n        .get(\"type_name\")\n        .cloned()\n        .or_else(|| metadata.get(\"bar_type\").map(|_| \"bars\".to_string()))\n        .ok_or_else(|| anyhow::anyhow!(\"Missing type_name in metadata\"))?;\n\n    match type_name.as_str() {\n        \"QuoteTick\" | \"quotes\" => Ok(QuoteTick::decode_data_batch(metadata, batch)?),\n        \"TradeTick\" | \"trades\" => Ok(TradeTick::decode_data_batch(metadata, batch)?),\n        \"Bar\" | \"bars\" => Ok(Bar::decode_data_batch(metadata, batch)?),\n        \"OrderBookDelta\" | \"order_book_deltas\" => {\n            Ok(OrderBookDelta::decode_data_batch(metadata, batch)?)\n        }\n        \"OrderBookDepth10\" | \"order_book_depths\" => {\n            Ok(OrderBookDepth10::decode_data_batch(metadata, batch)?)\n        }\n        \"MarkPriceUpdate\" | \"mark_price_updates\" => {\n            Ok(MarkPriceUpdate::decode_data_batch(metadata, batch)?)\n        }\n        \"IndexPriceUpdate\" | \"index_price_updates\" => {\n            Ok(IndexPriceUpdate::decode_data_batch(metadata, batch)?)\n        }\n        \"OptionGreeks\" | \"option_greeks\" => Ok(OptionGreeks::decode_data_batch(metadata, batch)?),","sourceCodeStart":170,"sourceCodeEnd":206,"githubUrl":"https://github.com/nautechsystems/nautilus_trader/blob/18893faf8b356be3320add8de2f861b0b647cf06/crates/persistence/src/backend/custom.rs#L170-L206","documentation":"Thrown by decode_batch_to_data when the batch schema metadata contains neither a 'type_name' key nor a 'bar_type' key, so the decoder cannot determine which data type to reconstruct. The type_name stored in metadata at write time is what routes the batch to the correct decode_data_batch implementation.","triggerScenarios":"Reading a feather/parquet file whose schema metadata lacks type_name — typically a file not written by the nautilus catalog writer, or metadata stripped during file transformation/copy; also a hand-built RecordBatch passed to decode_custom_batches_to_data without the metadata.","commonSituations":"External tools (PyArrow/pandas round-trips) that drop Arrow schema metadata; older files written before type_name metadata was introduced; manually constructed batches in tests.","solutions":["Re-write the file through the nautilus catalog writer so type_name metadata is embedded.","If the file only has 'bar_type' in metadata, confirm you're on a version whose fallback handles it (this code maps bar_type -> bars).","Manually add schema metadata: Schema::new_with_metadata(fields, [(\"type_name\", \"...\")]) before decoding.","Check for intermediate steps (compression, rewriting) that strip schema metadata and preserve it there."],"exampleFix":"// before\nlet batch = RecordBatch::try_new(schema, columns)?; // schema has no metadata\n// after\nlet mut metadata = HashMap::new();\nmetadata.insert(\"type_name\".to_string(), \"MyCustomData\".to_string());\nlet schema = Arc::new(Schema::new_with_metadata(fields.clone(), metadata));\nlet batch = RecordBatch::try_new(schema, columns)?;","handlingStrategy":"validation","validationCode":"anyhow::ensure!(metadata.contains_key(\"type_name\") || metadata.contains_key(\"bar_type\"), \"batch metadata missing type_name\");","typeGuard":"fn has_type_name(meta: &HashMap<String, String>) -> bool {\n    meta.contains_key(\"type_name\") || meta.contains_key(\"bar_type\")\n}","tryCatchPattern":"match decode_batch_to_data(&metadata, batch, true) {\n    Ok(data) => use(data),\n    Err(e) if e.to_string().contains(\"Missing type_name\") => reingest_or_rewrite_file()?,\n    Err(e) => return Err(e),\n}","preventionTips":["Avoid PyArrow/pandas round-trips that drop Arrow schema metadata","Always write catalog files through the nautilus writer","Verify schema.metadata() after any file transformation","Inject type_name metadata manually when constructing batches in tests"],"tags":["rust","persistence","metadata","custom-data"],"backgroundTag":"missing-required-config-field","analyzedSha":"18893faf8b356be3320add8de2f861b0b647cf06","analyzedAt":"2026-09-08T20:49:34.690Z","contentChangedAt":"2026-09-08T20:49:34.690Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}