{"record":{"id":"ce7a4fba37770151","repo":"nautechsystems/nautilus_trader","slug":"ts-init-column-is-not-uint64","errorCode":null,"errorMessage":"ts_init column is not UInt64","messagePattern":"ts_init column is not UInt64","errorType":"exception","errorClass":"anyhow::Error","httpStatus":null,"severity":"error","filePath":"crates/persistence/src/backend/catalog.rs","lineNumber":4152,"sourceCode":"        }\n\n        if ts_init.null_count() > 0 {\n            anyhow::bail!(\"ts_init column contains null values\");\n        }\n\n        Ok((ts_init.value(0), ts_init.value(ts_init.len() - 1)))\n    }\n\n    fn ts_init_array(batch: &RecordBatch) -> anyhow::Result<&UInt64Array> {\n        let ts_init_idx = batch\n            .schema()\n            .index_of(\"ts_init\")\n            .map_err(|_| anyhow::anyhow!(\"ts_init column not found\"))?;\n        batch\n            .column(ts_init_idx)\n            .as_any()\n            .downcast_ref::<UInt64Array>()\n            .ok_or_else(|| anyhow::anyhow!(\"ts_init column is not UInt64\"))\n    }\n\n    fn identifier_from_batch_or_path(\n        batch: &RecordBatch,\n        data_name: &str,\n        feather_path: &str,\n    ) -> Option<String> {\n        let metadata = batch.schema().metadata().clone();\n        if let Some(bar_type) = metadata.get(\"bar_type\") {\n            return Some(bar_type.clone());\n        }\n\n        if let Some(instrument_id) = metadata.get(\"instrument_id\") {\n            return Some(instrument_id.clone());\n        }\n\n        let parts: Vec<&str> = feather_path.trim_matches('/').split('/').collect();\n        if let Some(type_name) = data_name.strip_prefix(\"custom/\") {","sourceCodeStart":4134,"sourceCodeEnd":4170,"githubUrl":"https://github.com/nautechsystems/nautilus_trader/blob/18893faf8b356be3320add8de2f861b0b647cf06/crates/persistence/src/backend/catalog.rs#L4134-L4170","documentation":"This error is thrown when the Arrow RecordBatch column named 'ts_init' cannot be downcast to a UInt64Array. The persistence catalog expects ts_init (nanosecond UNIX timestamps) to always be UInt64; any other Arrow type (Int64, Timestamp, etc.) makes the downcast fail. It guards the invariant that time columns are stored as unsigned 64-bit integers.","triggerScenarios":"Calling ts_init_array on a RecordBatch whose 'ts_init' column was created with a non-UInt64 Arrow type, e.g. Int64Array or TimestampNanosecondArray. Typically happens when a custom encoder or an external writer produced the feather/parquet file with a differently-typed ts_init column.","commonSituations":"Custom data types with hand-written Arrow encoders; files written by an older nautilus version with a different ts_init representation; batches constructed in Python/PyArrow with Int64 ts_init then passed to the Rust catalog.","solutions":["Fix the producer so the ts_init column is built as UInt64Array (e.g. arrow::array::UInt64Array::from(...), or pa.uint64() in PyArrow).","If the column is another integer type, cast it to UInt64 before passing the batch to the catalog (RecordBatch with cast column).","Inspect the actual schema (batch.schema()) to confirm the ts_init field type and check for renamed/misordered columns that shadow ts_init.","Re-export or regenerate affected feather/parquet files written by the misbehaving writer."],"exampleFix":"// before\nlet ts_init = Int64Array::from(values); // downcast to UInt64Array fails\n// after\nlet ts_init = UInt64Array::from(values.iter().map(|v| *v as u64).collect::<Vec<_>>());","handlingStrategy":"validation","validationCode":"fn assert_ts_init_uint64(batch: &RecordBatch) -> anyhow::Result<()> {\n    let idx = batch.schema().index_of(\"ts_init\")?;\n    let field = batch.schema().field(idx);\n    anyhow::ensure!(matches!(field.data_type(), arrow::datatypes::DataType::UInt64), \"ts_init must be UInt64\");\n    Ok(())\n}","typeGuard":"fn is_uint64_col(batch: &RecordBatch, name: &str) -> bool {\n    batch.schema().field_with_name(name).map(|f| f.data_type() == &arrow::datatypes::DataType::UInt64).unwrap_or(false)\n}","tryCatchPattern":"match ts_init_array(&batch) {\n    Ok(col) => use(col),\n    Err(e) if e.to_string().contains(\"not UInt64\") => recast_and_retry(&batch)?,\n    Err(e) => return Err(e),\n}","preventionTips":["Always build ts_init columns with UInt64Array","In PyArrow use pa.uint64() for ts_init, never int64","Assert column types right after encoding custom data","Add schema checks in tests for every custom data encoder"],"tags":["arrow","rust","persistence","type-mismatch"],"backgroundTag":"type-mismatch","analyzedSha":"18893faf8b356be3320add8de2f861b0b647cf06","analyzedAt":"2026-09-08T20:49:34.690Z","contentChangedAt":"2026-09-08T20:49:34.690Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}