python/cpython · error · ValueError

readinto returned {n} outside buffer size {len(b)}

Error message

readinto returned {n} outside buffer size {len(b)}

What it means

Raised by IOBase.read (Lib/_pyio.py:625) after it allocates a bytearray of `size` and delegates to readinto(b): the raw stream's readinto must return either None (non-blocking, no data) or an integer 0 <= n <= len(b). A return value outside that range violates the readinto contract, so the io layer raises ValueError rather than return corrupted data.

Source

Thrown at Lib/_pyio.py:625

    # primitive operation, but that would lead to nasty recursion in case
    # a subclass doesn't implement either.)

    def read(self, size=-1):
        """Read and return up to size bytes, where size is an int.

        Returns an empty bytes object on EOF, or None if the object is
        set not to block and has no data to read.
        """
        if size is None:
            size = -1
        if size < 0:
            return self.readall()
        b = bytearray(size.__index__())
        n = self.readinto(b)
        if n is None:
            return None
        if n < 0 or n > len(b):
            raise ValueError(f"readinto returned {n} outside buffer size {len(b)}")
        del b[n:]
        return b.take_bytes()

    def readall(self):
        """Read until EOF, using multiple read() call."""
        res = bytearray()
        while data := self.read(DEFAULT_BUFFER_SIZE):
            res += data
        if res:
            return res.take_bytes()
        else:
            # b'' or None
            return data

    def readinto(self, b):
        """Read bytes into a pre-allocated bytes-like object b.

        Returns an int representing the number of bytes read (0 for EOF), or

View on GitHub (pinned to bc6749cc3b)

Solutions

1. Audit the custom readinto(): it must copy at most len(buffer) bytes and return the exact number of bytes actually written.
2. Return 0 to signal EOF and None only for non-blocking streams with no data available.
3. In tests, configure mocks explicitly: `mock_stream.readinto.return_value = 0`.
4. Inherit from io.RawIOBase / io.IOBase and reuse their read() default, which routes through a correct readinto or vice versa.

Example fix

# before
class MemRaw(io.RawIOBase):
    def readinto(self, b):
        data = self._src.read(len(b))
        b[:len(data)] = data
        return len(self._src)  # wrong: total length, not bytes copied

# after
class MemRaw(io.RawIOBase):
    def readinto(self, b):
        data = self._src.read(len(b))
        b[:len(data)] = data
        return len(data)  # exact count of bytes written into b

n = raw.readinto(b)
if n is not None and not (0 <= n <= len(b)):
    raise ValueError(f'custom readinto misbehaved: {n}')  # fail fast with context

try:
    data = raw.read(4096)
except ValueError as e:
    if 'readinto returned' in str(e):
        raise RuntimeError('bug in custom raw stream: readinto contract violated') from e
    raise

When it happens

Trigger: A custom raw stream class whose readinto() returns the wrong variable (e.g. total bytes of the source instead of bytes copied into the buffer); readinto returning -1 for EOF; readinto writing into the buffer but returning a stale/hardcoded length; a subclass of io.RawIOBase that skips the base implementation.

Common situations: Implementing in-memory or socket-based RawIOBase subclasses for testing, mocking, or wrapping proprietary transports; porting C extension streams where the return convention differs; mocking readinto in tests with a MagicMock that returns a Mock, not an int.

Related errors

• seek() returned an invalid position
• tell() returned an invalid position
• can't have text and binary mode at once
• can't have read/write/append mode at once
• must have exactly one of read/write/append mode

AI-assisted analysis of python/cpython@bc6749cc3b (2026-08-14). Data as JSON: /api/errors/789da40637d055fd. Report an issue: GitHub.