{"record":{"id":"ac08164d4c8c7244","repo":"nautechsystems/nautilus_trader","slug":"failed-to-create-streamreader-e","errorCode":null,"errorMessage":"Failed to create StreamReader: {e}","messagePattern":"Failed to create StreamReader: (.+?)","errorType":"exception","errorClass":"anyhow::Error","httpStatus":null,"severity":"error","filePath":"crates/persistence/src/backend/catalog.rs","lineNumber":3819,"sourceCode":"    /// and returns all `RecordBatches` contained within it.\n    fn read_feather_file(&self, file_path: &str) -> anyhow::Result<Vec<RecordBatch>> {\n        use datafusion::arrow::ipc::reader::StreamReader;\n\n        let bytes = self.execute_async(async {\n            let path = ObjectPath::from(file_path);\n            let result = self.object_store.get(&path).await?;\n            let bytes = result.bytes().await?;\n            Ok::<_, anyhow::Error>(bytes)\n        })?;\n\n        if bytes.is_empty() {\n            return Ok(Vec::new());\n        }\n\n        // Read the Arrow IPC stream\n        let cursor = Cursor::new(bytes.as_ref());\n        let reader = StreamReader::try_new(cursor, None)\n            .map_err(|e| anyhow::anyhow!(\"Failed to create StreamReader: {e}\"))?;\n\n        // Read all batches\n        let mut batches = Vec::new();\n\n        for batch_result in reader {\n            let batch = batch_result.map_err(|e| anyhow::anyhow!(\"Failed to read batch: {e}\"))?;\n            batches.push(batch);\n        }\n\n        Ok(batches)\n    }\n\n    /// Converts `RecordBatches` to Data objects, optionally replacing `ts_init` with `ts_event`.\n    fn convert_record_batches_to_data<T>(\n        batches: Vec<RecordBatch>,\n        use_ts_event_for_ts_init: bool,\n    ) -> anyhow::Result<Vec<T>>\n    where","sourceCodeStart":3801,"sourceCodeEnd":3837,"githubUrl":"https://github.com/nautechsystems/nautilus_trader/blob/18893faf8b356be3320add8de2f861b0b647cf06/crates/persistence/src/backend/catalog.rs#L3801-L3837","documentation":"Raised in `read_feather_file` when `StreamReader::try_new` from the Arrow IPC crate cannot open a feather (Arrow IPC stream) file fetched from the object store. This happens during initial header/schema parsing, meaning the bytes are not a valid Arrow IPC stream before any batches are read.","triggerScenarios":"Calling `read_run_data` or `convert_stream_to_data` where the discovered `.feather` file's bytes fail IPC stream validation: file truncated/empty-but-nonzero, wrong format (e.g. a Parquet file named `.feather`, or an Arrow File-format file rather than stream format), or corrupted download from the object store.","commonSituations":"Stream files written by an older Nautilus version with a different IPC format; files renamed or converted incorrectly; interrupted uploads leaving partial bytes; pointing the catalog at a directory where the `.feather` extension was applied to non-feather data.","solutions":["Open the file with pyarrow (`pa.ipc.open_stream`) or arrow-rs to confirm it is a valid Arrow IPC stream; regenerate the file if not.","Verify the file was written in stream (not file/table) IPC format; re-export with the current writer if it is a legacy file-format file.","Check file size and re-download/re-write — truncated or partially-uploaded files fail header parsing.","Confirm the object store path in the error points at the intended file, not a misnamed or placeholder file.","If upgrading from an older release, re-run the stream→catalog conversion from the original raw data."],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":"// before conversion, sanity-check the file\nlet bytes = fetch(file_path)?;\nif bytes.len() < 8 || &bytes[..4] != b\"ARRO\".map(|b: u8| b) {} // prefer: try opening with pa.ipc.open_stream in a check tool\n","typeGuard":null,"tryCatchPattern":"match catalog.read_feather_file_result(path) {\n    Ok(batches) => batches,\n    Err(e) if e.to_string().contains(\"Failed to create StreamReader\") => {\n        // quarantine/rewrite the invalid feather file, then continue\n        quarantine(path); Vec::new()\n    }\n    Err(e) => return Err(e),\n}","preventionTips":["Write feather files only with the catalog's writer so the IPC stream format is guaranteed.","Verify object-store uploads completed (size/E-tag check) before converting.","Validate legacy files with pyarrow pa.ipc.open_stream before migration.","Never rename non-feather files to .feather to make discovery pick them up."],"tags":["arrow","ipc","feather","persistence"],"backgroundTag":"file-read-failed","analyzedSha":"18893faf8b356be3320add8de2f861b0b647cf06","analyzedAt":"2026-09-08T20:49:34.690Z","contentChangedAt":"2026-09-08T20:49:34.690Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}