nautechsystems/nautilus_trader · error · anyhow::Error

custom data type '{type_name}' is not registered with an Arr

Error message

custom data type '{type_name}' is not registered with an Arrow schema containing ts_init; call ensure_custom_data_registered::<T>() before querying

What it means

When querying custom data dynamically from the catalog, NautilusTrader looks up the type's registered Arrow schema via `CustomDataDecoder::get_schema` and requires it to contain a `ts_init` field. If the custom type was never registered on the Rust side (or its schema lacks `ts_init`), the query cannot proceed and this error is thrown, telling the caller to register the type first.

Source

Thrown at crates/persistence/src/backend/catalog.rs:2012

            f.into_iter()
                .map(|p| self.to_object_path(&p).map(|op| op.to_string()))
                .collect::<anyhow::Result<Vec<_>>>()?
        } else {
            self.list_parquet_files_with_criteria(&path_prefix, identifiers, start, end)?
        };

        if files.is_empty() {
            return Ok(Vec::new());
        }

        // Use CustomDataDecoder for all custom data. Pass type_name so decode can look up
        // the type when Parquet/DataFusion does not preserve schema metadata. Callers must
        // ensure Rust custom types are registered via ensure_custom_data_registered::<T>().
        let mut lookup_metadata = HashMap::new();
        lookup_metadata.insert("type_name".to_string(), type_name.to_string());
        let registered_schema = CustomDataDecoder::get_schema(Some(lookup_metadata));
        registered_schema.field_with_name("ts_init").map_err(|_| {
            anyhow::anyhow!(
                "custom data type '{type_name}' is not registered with an Arrow schema containing ts_init; \
                 call ensure_custom_data_registered::<T>() before querying"
            )
        })?;

        for file in files {
            let identifier = extract_identifier_from_path(&file);
            let safe_type_name = make_sql_safe_identifier(type_name);
            let safe_sql_identifier = make_sql_safe_identifier(&identifier);
            let safe_filename = extract_sql_safe_filename(&file);
            let table_name =
                format!("custom_{safe_type_name}_{safe_sql_identifier}_{safe_filename}");
            let resolved_path = self.resolve_path_for_datafusion(&file);
            let sql_query = build_query(&table_name, start, end, where_clause);

            // Use schemaless registration so DataFusion preserves the parquet file's
            // schema metadata (e.g. `bar_type`) on output batches, since the
            // explicit-schema variant strips per-batch metadata that decoders rely on.

View on GitHub (pinned to 18893faf8b)

Solutions

Call `ensure_custom_data_registered::<T>()` for the custom type before querying, in every process that performs the query.
Verify the type's Arrow schema includes a `ts_init` field (all nautilus data types must carry it).
Check the exact `type_name` string used in the query matches the registered class name.
If data was written by an older schema, re-register with the schema used at write time or rewrite the data.

Example fix

// before
let rows = catalog.query_custom_data_dynamic("MyIndicator", &filter)?;

// after
ensure_custom_data_registered::<MyIndicator>();
let rows = catalog.query_custom_data_dynamic("MyIndicator", &filter)?;

Defensive patterns

Strategy: validation

Validate before calling

ensure_custom_data_registered::<MyIndicator>();
// then verify schema contains ts_init via CustomDataDecoder::get_schema(Some(metadata))

Try / catch

match catalog.query_custom_data_dynamic(type_name, &filter) {
    Ok(rows) => rows,
    Err(e) if e.to_string().contains("not registered") => {
        // register the type and retry once
        ...
    }
    Err(e) => return Err(e),
}

Prevention

Call ensure_custom_data_registered::<T>() at process startup for every custom type queried
Keep ts_init in every custom data schema
Use the exact registered type_name in queries
Re-register after any schema change and migrate stored data accordingly

When it happens

Trigger: Calling `query_custom_data_dynamic` (or the delete/consolidate paths that share it) with a `type_name` whose Rust type was never passed through `ensure_custom_data_registered::<T>()`, or whose registered schema lacks the `ts_init` field.

Common situations: Querying data written by Python-only custom classes without registering the corresponding Rust type; running a query in a new process/binary where the `ensure_custom_data_registered::<T>()` call was never executed; schema drift where a redefined class dropped or renamed `ts_init`; Parquet/DataFusion stripping schema metadata so registration must be redone per session.

Understand the failure class

Background: 'Could not be found', 'does not exist', 'not found in database': the resource-not-found family when an ID, slug, key, or URI lookup comes back empty — this error's family across 20 libraries.

Related errors

AI-assisted analysis of nautechsystems/nautilus_trader@18893faf8b (2026-09-08). Data as JSON: /api/errors/df1220a4cf822a2c. Report an issue: GitHub.