{"record":{"id":"df1220a4cf822a2c","repo":"nautechsystems/nautilus_trader","slug":"custom-data-type-type-name-is-not-registered-w","errorCode":null,"errorMessage":"custom data type '{type_name}' is not registered with an Arrow schema containing ts_init; call ensure_custom_data_registered::<T>() before querying","messagePattern":"custom data type '(.+?)' is not registered with an Arrow schema containing ts_init; call ensure_custom_data_registered::<T>\\(\\) before querying","errorType":"exception","errorClass":"anyhow::Error","httpStatus":null,"severity":"error","filePath":"crates/persistence/src/backend/catalog.rs","lineNumber":2012,"sourceCode":"            f.into_iter()\n                .map(|p| self.to_object_path(&p).map(|op| op.to_string()))\n                .collect::<anyhow::Result<Vec<_>>>()?\n        } else {\n            self.list_parquet_files_with_criteria(&path_prefix, identifiers, start, end)?\n        };\n\n        if files.is_empty() {\n            return Ok(Vec::new());\n        }\n\n        // Use CustomDataDecoder for all custom data. Pass type_name so decode can look up\n        // the type when Parquet/DataFusion does not preserve schema metadata. Callers must\n        // ensure Rust custom types are registered via ensure_custom_data_registered::<T>().\n        let mut lookup_metadata = HashMap::new();\n        lookup_metadata.insert(\"type_name\".to_string(), type_name.to_string());\n        let registered_schema = CustomDataDecoder::get_schema(Some(lookup_metadata));\n        registered_schema.field_with_name(\"ts_init\").map_err(|_| {\n            anyhow::anyhow!(\n                \"custom data type '{type_name}' is not registered with an Arrow schema containing ts_init; \\\n                 call ensure_custom_data_registered::<T>() before querying\"\n            )\n        })?;\n\n        for file in files {\n            let identifier = extract_identifier_from_path(&file);\n            let safe_type_name = make_sql_safe_identifier(type_name);\n            let safe_sql_identifier = make_sql_safe_identifier(&identifier);\n            let safe_filename = extract_sql_safe_filename(&file);\n            let table_name =\n                format!(\"custom_{safe_type_name}_{safe_sql_identifier}_{safe_filename}\");\n            let resolved_path = self.resolve_path_for_datafusion(&file);\n            let sql_query = build_query(&table_name, start, end, where_clause);\n\n            // Use schemaless registration so DataFusion preserves the parquet file's\n            // schema metadata (e.g. `bar_type`) on output batches, since the\n            // explicit-schema variant strips per-batch metadata that decoders rely on.","sourceCodeStart":1994,"sourceCodeEnd":2030,"githubUrl":"https://github.com/nautechsystems/nautilus_trader/blob/18893faf8b356be3320add8de2f861b0b647cf06/crates/persistence/src/backend/catalog.rs#L1994-L2030","documentation":"When querying custom data dynamically from the catalog, NautilusTrader looks up the type's registered Arrow schema via `CustomDataDecoder::get_schema` and requires it to contain a `ts_init` field. If the custom type was never registered on the Rust side (or its schema lacks `ts_init`), the query cannot proceed and this error is thrown, telling the caller to register the type first.","triggerScenarios":"Calling `query_custom_data_dynamic` (or the delete/consolidate paths that share it) with a `type_name` whose Rust type was never passed through `ensure_custom_data_registered::<T>()`, or whose registered schema lacks the `ts_init` field.","commonSituations":"Querying data written by Python-only custom classes without registering the corresponding Rust type; running a query in a new process/binary where the `ensure_custom_data_registered::<T>()` call was never executed; schema drift where a redefined class dropped or renamed `ts_init`; Parquet/DataFusion stripping schema metadata so registration must be redone per session.","solutions":["Call `ensure_custom_data_registered::<T>()` for the custom type before querying, in every process that performs the query.","Verify the type's Arrow schema includes a `ts_init` field (all nautilus data types must carry it).","Check the exact `type_name` string used in the query matches the registered class name.","If data was written by an older schema, re-register with the schema used at write time or rewrite the data."],"exampleFix":"// before\nlet rows = catalog.query_custom_data_dynamic(\"MyIndicator\", &filter)?;\n\n// after\nensure_custom_data_registered::<MyIndicator>();\nlet rows = catalog.query_custom_data_dynamic(\"MyIndicator\", &filter)?;","handlingStrategy":"validation","validationCode":"ensure_custom_data_registered::<MyIndicator>();\n// then verify schema contains ts_init via CustomDataDecoder::get_schema(Some(metadata))","typeGuard":null,"tryCatchPattern":"match catalog.query_custom_data_dynamic(type_name, &filter) {\n    Ok(rows) => rows,\n    Err(e) if e.to_string().contains(\"not registered\") => {\n        // register the type and retry once\n        ...\n    }\n    Err(e) => return Err(e),\n}","preventionTips":["Call ensure_custom_data_registered::<T>() at process startup for every custom type queried","Keep ts_init in every custom data schema","Use the exact registered type_name in queries","Re-register after any schema change and migrate stored data accordingly"],"tags":["custom-data","arrow","catalog","registration","persistence"],"backgroundTag":"resource-not-found","analyzedSha":"18893faf8b356be3320add8de2f861b0b647cf06","analyzedAt":"2026-09-08T20:49:34.690Z","contentChangedAt":"2026-09-08T20:49:34.690Z","schemaVersion":2},"datasetVersion":"2026-09-14T00:17:10.932Z"}