What causes "invalid number of bytes to read" in python/cpython?

Use read() or read(-1) for read-to-EOF; use non-negative sizes otherwise.

python/cpython · error · ValueError

invalid number of bytes to read

Error message

invalid number of bytes to read

What it means

Raised by BufferedReader.read(size) when an explicit size is less than -1. In the io API, size=None and size=-1 both mean 'read to EOF'; any other negative value has no defined meaning and is rejected with ValueError before the read lock is taken.

Source

Thrown at Lib/_pyio.py:1085

        self._read_lock = Lock()

    def readable(self):
        return self.raw.readable()

    def _reset_read_buf(self):
        self._read_buf = b""
        self._read_pos = 0

    def read(self, size=None):
        """Read size bytes.

        Returns exactly size bytes of data unless the underlying raw IO
        stream reaches EOF or if the call would block in non-blocking
        mode. If size is negative, read until EOF or until read() would
        block.
        """
        if size is not None and size < -1:
            raise ValueError("invalid number of bytes to read")
        with self._read_lock:
            return self._read_unlocked(size)

    def _read_unlocked(self, n=None):
        nodata_val = b""
        empty_values = (b"", None)
        buf = self._read_buf
        pos = self._read_pos

        # Special case for when the number of bytes to read is unspecified.
        if n is None or n == -1:
            self._reset_read_buf()
            if hasattr(self.raw, 'readall'):
                chunk = self.raw.readall()
                if chunk is None:
                    return buf[pos:] or None
                else:
                    return buf[pos:] + chunk

View on GitHub (pinned to bc6749cc3b)

Solutions

Use read() or read(-1) for read-to-EOF; use non-negative sizes otherwise.
Clamp before calling: reader.read(max(0, remaining)) — and skip the call when remaining == 0.
Fix underflow in length arithmetic (loop condition should stop before remaining goes negative).

Example fix

# before
while remaining:
    chunk = reader.read(remaining)  # remaining can hit -2 -> ValueError

# after
while remaining > 0:
    chunk = reader.read(remaining)
    ...
    remaining -= len(chunk)

Defensive patterns

Strategy: validation

Validate before calling

def safe_read(reader, n):
    if n is None or n == -1:
        return reader.read()
    if n < 0:
        raise ValueError(f"bad read size: {n}")
    return reader.read(n)

Try / catch

try:
    chunk = reader.read(n)
except ValueError as e:
    if "invalid number of bytes" in str(e):
        chunk = reader.read(max(0, n))
    else:
        raise

Prevention

Loop with 'while remaining > 0', not 'while remaining'.
Map 'unlimited' sentinels to None/-1 exactly once at the boundary.
Never forward external sizes unchecked into read().

When it happens

Trigger: reader.read(-2), or read(n) where n is a negative sentinel from caller code (e.g. -1 used for 'unlimited' colliding with another -1-means-something-else convention, decremented past -1).

Common situations: Passing a 'remaining bytes' counter that has gone negative (reading past an expected length); forwarding a size from an API where negatives have different semantics; arithmetic like size - already_read producing < -1.

Related errors

AI-assisted analysis of python/cpython@bc6749cc3b (2026-08-14). Data as JSON: /api/errors/f48ee5dc10978952. Report an issue: GitHub.