nautechsystems/nautilus_trader · error

Stream advance error: {e}

Error message

Stream advance error: {e}

What it means

While iterating instrument definition records from a DBN zstd file, `dbn_stream.advance()` returned an Err — the underlying DBN decoder could not advance to the next record (I/O or decode failure). The adapter wraps it in anyhow and yields it from the returned iterator.

Source

Thrown at crates/adapters/databento/src/loader.rs:257

    /// # Errors
    ///
    /// Returns an error if decoding the definition records fails.
    pub fn read_definition_records<'a>(
        &'a mut self,
        filepath: &Path,
        use_exchange_as_venue: bool,
        decode_config: Option<&'a DatabentoDecodeConfig>,
    ) -> anyhow::Result<impl Iterator<Item = anyhow::Result<InstrumentAny>> + 'a> {
        let decoder = Decoder::from_zstd_file(filepath)?;
        let mut dbn_stream = decoder.decode_stream::<InstrumentDefMsg>();

        // Loop over skipped records (Ok(None)) so one unsupported class does not
        // terminate the stream
        Ok(std::iter::from_fn(move || {
            loop {
                let advance = dbn_stream
                    .advance()
                    .map_err(|e| anyhow::anyhow!("Stream advance error: {e}"));
                if let Err(e) = advance {
                    return Some(Err(e));
                }

                let rec = dbn_stream.get()?;

                let result: anyhow::Result<Option<InstrumentAny>> = (|| {
                    let record = dbn::RecordRef::from(rec);
                    let msg = record
                        .get::<InstrumentDefMsg>()
                        .ok_or_else(|| anyhow::anyhow!("Failed to decode InstrumentDefMsg"))?;

                    let raw_symbol = rec
                        .raw_symbol()
                        .map_err(|e| anyhow::anyhow!("Error decoding `raw_symbol`: {e}"))?;
                    let symbol = Symbol::from(raw_symbol);

                    let publisher = rec

View on GitHub (pinned to 18893faf8b)

Solutions

  1. Re-download the definition file from Databento; verify integrity (file size / checksum)
  2. Ensure the `databento-dbn` crate version is recent enough for the file's DBN schema version
  3. Test the file decodes with the official dbn CLI/tools to isolate adapter vs data problem
  4. Point read_definition_records at the correct file path — passing a data-schema file where a definition file is expected also breaks the stream
Defensive patterns

Strategy: validation

Validate before calling

// verify the file decodes and is a definition schema before iterating
let schema = loader.schema_from_file(&path)?;
if schema.as_deref() != Some("definition") {
    anyhow::bail!("expected definition schema, got {schema:?}");
}

Try / catch

let mut it = loader.read_definition_records(&path, true, None)?;
loop {
    match it.next() {
        Some(Ok(inst)) => { /* use inst */ }
        Some(Err(e)) if e.to_string().contains("Stream advance error") => {
            log::error!("corrupt definition file: {e}"); break;
        }
        Some(Err(e)) => return Err(e),
        None => break,
    }
}

Prevention

When it happens

Trigger: Calling `read_definition_records` (via load_instruments) on a zstd-compressed DBN file whose stream cannot be advanced: corrupt/truncated zstd frame, invalid DBN record stream mid-file, or I/O error beneath the decoder.

Common situations: Partially downloaded or interrupted Databento batch download, file written with a newer DBN format version than the decoder supports, disk read errors.

Understand the failure class

Background: "failed to read file", EACCES, ENOENT and "could not read <path>" errors: when a program can't read a file from disk — this error's family across 49 libraries.

Related errors

AI-assisted analysis of nautechsystems/nautilus_trader@18893faf8b (2026-09-08). Data as JSON: /api/errors/eda3c24230cecb1e. Report an issue: GitHub.