{"record":{"id":"4ca72e8a9be9872f","repo":"nautechsystems/nautilus_trader","slug":"e-to-string","errorCode":null,"errorMessage":"e.to_string()","messagePattern":"e\\.to_string\\(\\)","errorType":"exception","errorClass":"anyhow::Error","httpStatus":null,"severity":"error","filePath":"crates/persistence/src/backend/catalog.rs","lineNumber":2038,"sourceCode":"            let safe_type_name = make_sql_safe_identifier(type_name);\n            let safe_sql_identifier = make_sql_safe_identifier(&identifier);\n            let safe_filename = extract_sql_safe_filename(&file);\n            let table_name =\n                format!(\"custom_{safe_type_name}_{safe_sql_identifier}_{safe_filename}\");\n            let resolved_path = self.resolve_path_for_datafusion(&file);\n            let sql_query = build_query(&table_name, start, end, where_clause);\n\n            // Use schemaless registration so DataFusion preserves the parquet file's\n            // schema metadata (e.g. `bar_type`) on output batches, since the\n            // explicit-schema variant strips per-batch metadata that decoders rely on.\n            self.session\n                .add_file::<CustomDataDecoder>(\n                    &table_name,\n                    &resolved_path,\n                    Some(&sql_query),\n                    Some(type_name),\n                )\n                .map_err(|e| anyhow::anyhow!(e.to_string()))?;\n        }\n\n        let query_result = self.session.get_query_result();\n        Ok(query_result.collect::<Result<Vec<_>, _>>()?)\n    }\n\n    /// Queries all Parquet files for a specific data type and optional instrument IDs.\n    ///\n    /// This method finds all Parquet files that match the specified criteria and returns\n    /// their full URIs. The files are filtered by data type, instrument IDs (if provided),\n    /// and timestamp range (if provided).\n    ///\n    /// # Parameters\n    ///\n    /// - `data_cls`: The data type directory name (e.g., \"quotes\", \"trades\").\n    /// - `identifiers`: Optional list of identifiers to filter by. Can be `instrument_id` strings\n    ///   (e.g., \"EUR/USD.SIM\") or `bar_type` strings (e.g., \"EUR/USD.SIM-1-MINUTE-LAST-EXTERNAL\").\n    ///   For bars, partial matching is supported.","sourceCodeStart":2020,"sourceCodeEnd":2056,"githubUrl":"https://github.com/nautechsystems/nautilus_trader/blob/18893faf8b356be3320add8de2f861b0b647cf06/crates/persistence/src/backend/catalog.rs#L2020-L2056","documentation":"Raised in `query_custom_data_dynamic` when registering a Parquet file with the DataFusion-backed query session via `add_file::<CustomDataDecoder>` fails. The underlying error (usually a DataFusion/Parquet registration or execution error) is stringified and re-wrapped in an `anyhow::Error`. The catalog needs each matching Parquet file registered as a table (with a bounded SQL query) before it can run the caller's query.","triggerScenarios":"Calling `query_custom_data_dynamic` (directly or via consolidate/delete range helpers) when one of the discovered custom-data Parquet files cannot be registered: the file is missing or unreadable at the resolved path, the file is corrupt or not valid Parquet, its schema is incompatible with the decoder, or the internal DataFusion session rejects the generated SQL/table name.","commonSituations":"Custom data files written by an older NautilusTrader version whose schema no longer matches the current `CustomDataDecoder`; truncated or partially-written Parquet files left by a crashed run; querying a custom type that was written under a different type_name than registered; object-store connectivity failures when resolving remote paths.","solutions":["Read the wrapped message (it is the original `e.to_string()`) to identify whether registration, schema, or file access failed, then fix that root cause.","Verify the custom data type is registered before querying: call `ensure_custom_data_registered::<T>()` for the Rust type being queried.","Check the Parquet files listed for the type are valid (e.g. open with pyarrow/parquet-tools) and re-write or remove corrupt files.","Re-check the catalog path/URI configuration so `resolve_path_for_datafusion` points at files that actually exist in the object store.","If the schema changed between versions, rewrite the affected custom-data files with the current writer so the schema matches the decoder."],"exampleFix":"// before: querying a custom type never registered\nlet rows = catalog.query_custom_data_dynamic(\"MyData\", None, None, None)?;\n\n// after: register the type first\ncatalog.ensure_custom_data_registered::<MyData>()?;\nlet rows = catalog.query_custom_data_dynamic(\"MyData\", None, None, None)?;","handlingStrategy":"try-catch","validationCode":"catalog.ensure_custom_data_registered::<MyData>()?;\n// and before querying, verify files are non-empty valid parquet:\n// e.g. open each listed file with an Arrow parquet reader and check schema fields contain ts_init/ts_event","typeGuard":null,"tryCatchPattern":"match catalog.query_custom_data_dynamic(type_name, None, None, None) {\n    Ok(rows) => rows,\n    Err(e) => { log::error!(\"custom data query failed: {e}\"); Vec::new() } // inspect stringified cause\n}","preventionTips":["Always call ensure_custom_data_registered::<T>() before querying custom data.","Validate written parquet files (open + schema check) before relying on them in queries.","Keep writer and reader on the same NautilusTrader version; migrate files after upgrades.","Check object-store connectivity/credentials before bulk queries."],"tags":["arrow","datafusion","parquet","persistence"],"backgroundTag":"database-query-failed","analyzedSha":"18893faf8b356be3320add8de2f861b0b647cf06","analyzedAt":"2026-09-08T20:49:34.690Z","contentChangedAt":"2026-09-08T20:49:34.690Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}