nautechsystems/nautilus_trader · error · anyhow::Error

Failed to deserialize record: {e}

Error message

Failed to deserialize record: {e}

What it means

After a CSV row is successfully read in `DeltaStreamIterator::read_record`, it is deserialized into `TardisBookUpdateRecord`. If the row's fields do not match the expected Tardis book-update schema (missing/invalid columns, wrong types, unexpected values), the deserializer error is wrapped as this error. It means the file is readable but its content does not conform to the expected record shape.

Source

Thrown at crates/adapters/tardis/src/csv/stream.rs:311

            Some(Ok(self.buffer.clone()))
        }
    }
}

impl DeltaStreamIterator {
    fn read_record(&mut self) -> anyhow::Result<Option<TardisBookUpdateRecord>> {
        if !self
            .reader
            .read_record(&mut self.record)
            .map_err(|e| anyhow::anyhow!("Failed to read record: {e}"))?
        {
            return Ok(None);
        }

        self.record
            .deserialize::<TardisBookUpdateRecord>(None)
            .map(Some)
            .map_err(|e| anyhow::anyhow!("Failed to deserialize record: {e}"))
    }
}

/// Streams [`OrderBookDelta`]s from a Tardis format CSV at the given `filepath`,
/// yielding chunks of the specified size.
///
/// # Precision Inference Warning
///
/// When using streaming with precision inference (not providing explicit precisions),
/// the inferred precision may differ from bulk loading the entire file. This is because
/// precision inference works within chunk boundaries, and different chunks may contain
/// values with different precision requirements. For deterministic precision behavior,
/// provide explicit `price_precision` and `size_precision` parameters.
///
/// # Errors
///
/// Returns an error if `chunk_size` is outside `[1, 1_000_000]`, or if the file cannot be opened,
/// read, or parsed as CSV.

View on GitHub (pinned to 18893faf8b)

Solutions

  1. Confirm the file is a Tardis incremental book (derivative_ticker/book) CSV, not trades or options-chain output.
  2. Open the failing row and compare columns against the expected `TardisBookUpdateRecord` schema (exchange, symbol, side, price, size, timestamp, local_timestamp...).
  3. Re-export the data with the current Tardis schema if the file came from an older exporter version.
  4. Check for NaN/empty numeric fields; fix or drop malformed rows before streaming.

Example fix

// before: wrong file kind
let stream = DeltaStream::from_path("trades.csv", ...)?;
// after: use the matching reader for the file kind
if is_book_csv(path) { DeltaStream::from_path(path, ...) } else { OptionsChainStream::from_path(path, ...) }
Defensive patterns

Strategy: validation

Validate before calling

REQUIRED_BOOK_COLS = {"exchange", "symbol", "side", "price", "size", "timestamp", "local_timestamp"}

def validate_book_csv_header(path: str) -> None:
    import csv
    with open(path, newline="") as f:
        header = set(next(csv.reader(f)))
    missing = REQUIRED_BOOK_COLS - header
    if missing:
        raise ValueError(f"{path}: not a Tardis book CSV, missing {sorted(missing)}")

Type guard

def is_book_csv(path: str) -> bool:
    import csv
    with open(path, newline="") as f:
        return REQUIRED_BOOK_COLS.issubset(set(next(csv.reader(f))))

Try / catch

try:
    for batch in stream:
        process(batch)
except Exception as e:
    if "Failed to deserialize record" in str(e):
        raise DataSchemaError(f"CSV does not match TardisBookUpdateRecord schema: {e}") from e
    raise

Prevention

When it happens

Trigger: Calling the delta stream `next` on a CSV whose rows deviate from the Tardis book update schema: missing required fields (e.g. `symbol`, `timestamp`), non-numeric values in numeric columns, or a file exported by a different Tardis endpoint/version with a changed schema.

Common situations: Pointing the stream at an options-chain or trades CSV instead of a book updates CSV; a Tardis schema/version change; hand-edited or filtered CSVs that dropped columns.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

AI-assisted analysis of nautechsystems/nautilus_trader@18893faf8b (2026-09-08). Data as JSON: /api/errors/e826c711bf16775c. Report an issue: GitHub.