What causes "could not read from input" in pola-rs/polars?

Fix the object's read() to return bytes (e.g. return bytes(chunk) instead of bytearray/memoryview)

pola-rs/polars · error · PolarsError

InvalidOperation

InvalidOperation

Error message

could not read from input

What it means

polars-python's Read impl for PyFileLikeObject calls .read(n) on the Python file-like object you passed and requires the result to be exactly bytes or str. If read() returns any other type (None, int, bytearray, memoryview, custom class), this InvalidInput error is raised. The upfront ensure_requirements check only verifies that a read method exists, not what it returns.

Source

Thrown at crates/polars-python/src/file.rs:179

    fn read(&mut self, mut buf: &mut [u8]) -> Result<usize, io::Error> {
        Python::attach(|py| {
            let bytes = self
                .inner
                .call_method(py, "read", (buf.len(),), None)
                .map_err(pyerr_to_io_err)?;

            let opt_bytes = bytes.cast_bound::<PyBytes>(py);

            if let Ok(bytes) = opt_bytes {
                buf.write_all(bytes.as_bytes())?;

                bytes.len().map_err(pyerr_to_io_err)
            } else if let Ok(s) = bytes.cast_bound::<PyString>(py) {
                let s = s.to_cow().map_err(pyerr_to_io_err)?;
                buf.write_all(s.as_bytes())?;
                Ok(s.len())
            } else {
                Err(io::Error::new(
                    ErrorKind::InvalidInput,
                    polars_err!(InvalidOperation: "could not read from input"),
                ))
            }
        })
    }
}

impl Write for PyFileLikeObject {
    fn write(&mut self, buf: &[u8]) -> Result<usize, io::Error> {
        // Note on the .extract() method:
        // In case of a PyString object, it returns the number of chars,
        // so we need to take extra steps if the underlying string is not all ASCII.
        // In case of a ByBytes object, it returns the number of bytes.
        let expects_str = self.expects_str;
        let expects_str_and_is_ascii = expects_str && buf.is_ascii();

        Python::attach(|py| {

View on GitHub (pinned to df599052da)

Solutions

Fix the object's read() to return bytes (e.g. return bytes(chunk) instead of bytearray/memoryview)
Wrap the source so read always yields bytes: class BytesAdapter: def read(self, n): return bytes(self.inner.read(n))
Buffer the whole source into io.BytesIO and pass that instead
Pass a real filesystem path or URL instead of a file-like object

Example fix

# before
class MyStream:
    def read(self, n):
        return self.sock.recv(n)  # may return bytearray/memoryview

pl.read_csv(source=MyStream())  # InvalidOperation: could not read from input

# after
class MyStream:
    def read(self, n):
        return bytes(self.sock.recv(n))

pl.read_csv(source=MyStream())

Defensive patterns

Strategy: type-guard

Validate before calling

data = obj.read(0) if callable(getattr(obj, "read", None)) else None
if not isinstance(data, (bytes, str)):
    raise TypeError("read() must return bytes or str, got %r" % type(data))
pl.read_csv(source=obj)

Type guard

def is_polars_readable(obj) -> bool:
    """True if obj.read() returns bytes or str (what polars accepts)."""
    read = getattr(obj, "read", None)
    if not callable(read):
        return False
    probe = read(0)
    return probe is None or isinstance(probe, (bytes, str))

Prevention

Prefer io.BytesIO or real paths over custom file-like objects
In adapters, coerce read() results: return bytes(chunk)
Return b'' at EOF, never None or a count
Probe obj.read(0) once before handing the object to polars

When it happens

Trigger: Passing a custom file-like object to read_csv/read_ipc/scan_* whose read() returns bytearray, memoryview, None, or a wrapper object instead of bytes/str.

Common situations: Adapters over network or database streams yielding bytearray chunks; objects mimicking io.IOBase but returning memoryview; wrappers that return a byte count instead of the data; read() overridden to return None at EOF instead of b''.

Related errors

AI-assisted analysis of pola-rs/polars@df599052da (2026-08-16). Data as JSON: /api/errors/8221b0f7bb81b61b. Report an issue: GitHub.