nautechsystems/nautilus_trader · error · anyhow::Error

Failed to read record: {e}

Error message

Failed to read record: {e}

What it means

`DeltaStreamIterator::read_record` reads raw CSV rows via the csv reader. If the underlying reader returns an I/O or CSV parsing error, it is wrapped in this error rather than being silently dropped, so the streaming consumer can surface the failure. It indicates the file could not be read at the record level — distinct from deserialization failures of an otherwise readable row.

Source

Thrown at crates/adapters/tardis/src/csv/stream.rs:303

            // Only set F_LAST when limit reached (stream ending), not on chunk
            // boundary where more deltas from the same message may follow
            if let Some(limit) = self.limit
                && self.deltas_emitted >= limit
                && let Some(last_delta) = self.buffer.last_mut()
            {
                last_delta.flags = RecordFlag::F_LAST as u8;
            }
            Some(Ok(self.buffer.clone()))
        }
    }
}

impl DeltaStreamIterator {
    fn read_record(&mut self) -> anyhow::Result<Option<TardisBookUpdateRecord>> {
        if !self
            .reader
            .read_record(&mut self.record)
            .map_err(|e| anyhow::anyhow!("Failed to read record: {e}"))?
        {
            return Ok(None);
        }

        self.record
            .deserialize::<TardisBookUpdateRecord>(None)
            .map(Some)
            .map_err(|e| anyhow::anyhow!("Failed to deserialize record: {e}"))
    }
}

/// Streams [`OrderBookDelta`]s from a Tardis format CSV at the given `filepath`,
/// yielding chunks of the specified size.
///
/// # Precision Inference Warning
///
/// When using streaming with precision inference (not providing explicit precisions),
/// the inferred precision may differ from bulk loading the entire file. This is because

View on GitHub (pinned to 18893faf8b)

Solutions

1. Verify the file path exists and is readable: `ls -l`, check permissions, and re-test locally.
2. Re-download or re-export the CSV; check for truncation or corruption (compare file size/checksum with the source).
3. Confirm the file is standard comma-separated UTF-8 CSV as produced by Tardis Machine; re-export if quoting or encoding differs.
4. Retry the read if the file lives on a flaky network mount.

Example fix

// before
let stream = DeltaStream::from_path("/mnt/net/tardis.csv", ...)?; // flaky mount
// after: verify and fail fast before streaming
let file = std::fs::File::open("/mnt/net/tardis.csv")?;
let len = file.metadata()?.len();
assert!(len > 0, "CSV truncated");
let stream = DeltaStream::from_path("/mnt/net/tardis.csv", ...)?;

from pathlib import Path

def ensure_readable_csv(path: str) -> None:
    p = Path(path)
    if not p.is_file():
        raise FileNotFoundError(path)
    if p.stat().st_size == 0:
        raise ValueError(f"{path} is empty (truncated download?)")
    with open(p, "rb") as f:
        f.read(1024).decode("utf-8")  # raises on invalid encoding

def is_readable_csv(path: str) -> bool:
    from pathlib import Path
    p = Path(path)
    return p.is_file() and p.stat().st_size > 0

try:
    for batch in stream:
        process(batch)
except Exception as e:
    if "Failed to read record" in str(e):
        # I/O-level failure: verify file, re-download, retry once from local copy
        local = stage_local(path)
        retry_stream(local)
    else:
        raise

When it happens

Trigger: Iterating a `DeltaStream` whose `next` calls `read_record` while the CSV file is unreadable, deleted mid-read, on a failing disk/network mount, malformed at the byte level (e.g. wrong quoting/delimiter so the csv crate errors), or not valid UTF-8.

Common situations: Reading a file over an unstable network share; a truncated download; a file written with a different delimiter or quoting convention than the csv reader expects.

Understand the failure class

Background: "failed to read file", EACCES, ENOENT and "could not read <path>" errors: when a program can't read a file from disk — this error's family across 49 libraries.

Related errors

• Failed to reset file position for '{}' after {MAX_RETRIES} a
• snapshot interval must be positive
• options_chain CSV rows must be ordered by local_timestamp wh
• Invalid delta: action {action} when size zero, check size_pr
• Failed to deserialize record: {e}

AI-assisted analysis of nautechsystems/nautilus_trader@18893faf8b (2026-09-08). Data as JSON: /api/errors/051f6a34baa5dbc0. Report an issue: GitHub.