{"record":{"id":"a530393e4a4ea12e","repo":"pola-rs/polars","slug":"could-not-read","errorCode":null,"errorMessage":"could not read","messagePattern":"could not read","errorType":"panic","errorClass":null,"httpStatus":null,"severity":"error","filePath":"crates/polars-io/src/mmap.rs","lineNumber":101,"sourceCode":"\nimpl<'a, T: 'a + MmapBytesReader> From<&'a mut T> for ReaderBytes<'a> {\n    fn from(m: &'a mut T) -> Self {\n        match m.to_bytes() {\n            // , but somehow bchk doesn't see that lifetime is 'a.\n            Some(s) => {\n                let s = unsafe { std::mem::transmute::<&[u8], &'a [u8]>(s) };\n                ReaderBytes::Borrowed(s)\n            },\n            None => {\n                if let Some(f) = m.to_file() {\n                    let mmap = MMapSemaphore::new_from_file(f).unwrap();\n                    ReaderBytes::Owned(Buffer::from_owner(mmap))\n                } else {\n                    if verbose() {\n                        eprintln!(\"could not memory map file; read to buffer.\")\n                    }\n                    let mut buf = vec![];\n                    m.read_to_end(&mut buf).expect(\"could not read\");\n                    ReaderBytes::Owned(Buffer::from_vec(buf))\n                }\n            },\n        }\n    }\n}\n","sourceCodeStart":83,"sourceCodeEnd":108,"githubUrl":"https://github.com/pola-rs/polars/blob/9b5d73fd00236295624374b075d16b1fe6ec6df9/crates/polars-io/src/mmap.rs#L83-L108","documentation":"ReaderBytes::from tries three strategies in order: borrow the bytes (Cursor-like sources), mmap the file, or stream-read into a buffer. The .expect(\"could not read\") fires on the last resort: the source exposed neither bytes nor a File handle (typical for compressed/wrapped readers), and read_to_end then returned an I/O error. So the panic means a real OS-level read failure on a non-mmap-able source, surfaced as a panic instead of an io::Result.","triggerScenarios":"read_csv/scan over a source that has no File handle (e.g. a gzip/zstd compressed reader, a pipe, or process substitution like <(cat x.gz)) whose stream fails mid-copy: corrupt archive (unexpected EOF), EPIPE from a closed producer, EIO on failing disks/NFS, or permission changes after open.","commonSituations":"Partially downloaded .gz files; a plain CSV mislabeled with a .gz extension so the decompressing reader hits garbage; named pipes whose writer exits early; flaky network mounts.","solutions":["Verify the source out-of-band first: gzip -t data.csv.gz, and re-stage the file if truncated","Read to memory yourself and hand polars bytes so you get a real io::Result: let bytes = std::fs::read(path)?; CsvReadOptions::default().parse(Cursor::new(bytes))?","If reading from pipes/fifos, buffer to a temp file or Vec first - the mmap fast path needs a real file anyway","Check dmesg/NFS logs for the underlying EIO if the file is local and integrity checks pass"],"exampleFix":"// before (panic inside read_to_end on a corrupt/truncated stream)\nlet df = CsvReadOptions::default().parse(path)?;\n\n// after (io::Error you can actually handle)\nlet bytes = std::fs::read(path)?;   // or fetch over HTTP and validate first\nlet df = CsvReadOptions::default().parse(std::io::Cursor::new(bytes))?;","handlingStrategy":"fallback","validationCode":"use std::io::{Read, Seek, SeekFrom};\nfn preflight_readable(path: &std::path::Path) -> std::io::Result<()> {\n    let mut f = std::fs::File::open(path)?;\n    let meta = f.metadata()?;\n    if meta.len() == 0 { return Ok(()); }\n    let mut probe = vec![0u8; 4096.min(meta.len() as usize)];\n    f.read_exact(&mut probe)?;   // surfaces permission/EIO errors before polars\n    f.seek(SeekFrom::Start(0))?;\n    Ok(())\n}\n// then prefer handing polars bytes you read yourself:\n// let bytes = std::fs::read(path)?; CsvReadOptions::default().parse(Cursor::new(bytes))?;","typeGuard":null,"tryCatchPattern":"catch_unwind(AssertUnwindSafe(|| CsvReadOptions::default().parse(path.clone()))) to log 'source unreadable: {path}'; the stream is dead so only retry after re-staging the file (re-download, gzip -t).","preventionTips":["Checksum downloads (sha256 / gzip -t) before handing paths to polars","Buffer untrusted or compressed inputs into memory yourself so I/O errors come back as io::Result","Monitor ingest for truncated files; fail the batch instead of the process"],"tags":["polars","io","mmap","filesystem","compression","corrupt-file","panic"],"backgroundTag":"file-read-io-error","analyzedSha":"9b5d73fd00236295624374b075d16b1fe6ec6df9","analyzedAt":"2026-08-19T12:15:06.350Z","contentChangedAt":"2026-08-19T12:15:06.350Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}