nautechsystems/nautilus_trader · error · anyhow::Error

ts_init column is not UInt64

Error message

ts_init column is not UInt64

What it means

This error is thrown when the Arrow RecordBatch column named 'ts_init' cannot be downcast to a UInt64Array. The persistence catalog expects ts_init (nanosecond UNIX timestamps) to always be UInt64; any other Arrow type (Int64, Timestamp, etc.) makes the downcast fail. It guards the invariant that time columns are stored as unsigned 64-bit integers.

Source

Thrown at crates/persistence/src/backend/catalog.rs:4152

        }

        if ts_init.null_count() > 0 {
            anyhow::bail!("ts_init column contains null values");
        }

        Ok((ts_init.value(0), ts_init.value(ts_init.len() - 1)))
    }

    fn ts_init_array(batch: &RecordBatch) -> anyhow::Result<&UInt64Array> {
        let ts_init_idx = batch
            .schema()
            .index_of("ts_init")
            .map_err(|_| anyhow::anyhow!("ts_init column not found"))?;
        batch
            .column(ts_init_idx)
            .as_any()
            .downcast_ref::<UInt64Array>()
            .ok_or_else(|| anyhow::anyhow!("ts_init column is not UInt64"))
    }

    fn identifier_from_batch_or_path(
        batch: &RecordBatch,
        data_name: &str,
        feather_path: &str,
    ) -> Option<String> {
        let metadata = batch.schema().metadata().clone();
        if let Some(bar_type) = metadata.get("bar_type") {
            return Some(bar_type.clone());
        }

        if let Some(instrument_id) = metadata.get("instrument_id") {
            return Some(instrument_id.clone());
        }

        let parts: Vec<&str> = feather_path.trim_matches('/').split('/').collect();
        if let Some(type_name) = data_name.strip_prefix("custom/") {

View on GitHub (pinned to 18893faf8b)

Solutions

1. Fix the producer so the ts_init column is built as UInt64Array (e.g. arrow::array::UInt64Array::from(...), or pa.uint64() in PyArrow).
2. If the column is another integer type, cast it to UInt64 before passing the batch to the catalog (RecordBatch with cast column).
3. Inspect the actual schema (batch.schema()) to confirm the ts_init field type and check for renamed/misordered columns that shadow ts_init.
4. Re-export or regenerate affected feather/parquet files written by the misbehaving writer.

Example fix

// before
let ts_init = Int64Array::from(values); // downcast to UInt64Array fails
// after
let ts_init = UInt64Array::from(values.iter().map(|v| *v as u64).collect::<Vec<_>>());

fn assert_ts_init_uint64(batch: &RecordBatch) -> anyhow::Result<()> {
    let idx = batch.schema().index_of("ts_init")?;
    let field = batch.schema().field(idx);
    anyhow::ensure!(matches!(field.data_type(), arrow::datatypes::DataType::UInt64), "ts_init must be UInt64");
    Ok(())
}

fn is_uint64_col(batch: &RecordBatch, name: &str) -> bool {
    batch.schema().field_with_name(name).map(|f| f.data_type() == &arrow::datatypes::DataType::UInt64).unwrap_or(false)
}

match ts_init_array(&batch) {
    Ok(col) => use(col),
    Err(e) if e.to_string().contains("not UInt64") => recast_and_retry(&batch)?,
    Err(e) => return Err(e),
}

When it happens

Trigger: Calling ts_init_array on a RecordBatch whose 'ts_init' column was created with a non-UInt64 Arrow type, e.g. Int64Array or TimestampNanosecondArray. Typically happens when a custom encoder or an external writer produced the feather/parquet file with a differently-typed ts_init column.

Common situations: Custom data types with hand-written Arrow encoders; files written by an older nautilus version with a different ts_init representation; batches constructed in Python/PyArrow with Int64 ts_init then passed to the Rust catalog.

Understand the failure class

Background: Type mismatch errors: IllegalArgumentException, TypeError and type guards across 150 open-source libraries — this error's family across 150 libraries.

Related errors

• Failed to create new batch: {e}
• Expected {}, was different type
• Expected {}
• Expected Custom data variant
• Failed to concatenate stream batches: {e}

AI-assisted analysis of nautechsystems/nautilus_trader@18893faf8b (2026-09-08). Data as JSON: /api/errors/ce7a4fba37770151. Report an issue: GitHub.