{"record":{"id":"13c12ca24bcda229","repo":"RustPython/RustPython","slug":"readinto-returned-n-outside-buffer-size-len-b","errorCode":null,"errorMessage":"readinto returned {n} outside buffer size {len(b)}","messagePattern":"readinto returned (.+?) outside buffer size (.+?)","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"Lib/_pyio.py","lineNumber":625,"sourceCode":"    # primitive operation, but that would lead to nasty recursion in case\n    # a subclass doesn't implement either.)\n\n    def read(self, size=-1):\n        \"\"\"Read and return up to size bytes, where size is an int.\n\n        Returns an empty bytes object on EOF, or None if the object is\n        set not to block and has no data to read.\n        \"\"\"\n        if size is None:\n            size = -1\n        if size < 0:\n            return self.readall()\n        b = bytearray(size.__index__())\n        n = self.readinto(b)\n        if n is None:\n            return None\n        if n < 0 or n > len(b):\n            raise ValueError(f\"readinto returned {n} outside buffer size {len(b)}\")\n        del b[n:]\n        return bytes(b)\n\n    def readall(self):\n        \"\"\"Read until EOF, using multiple read() call.\"\"\"\n        res = bytearray()\n        while data := self.read(DEFAULT_BUFFER_SIZE):\n            res += data\n        if res:\n            return bytes(res)\n        else:\n            # b'' or None\n            return data\n\n    def readinto(self, b):\n        \"\"\"Read bytes into a pre-allocated bytes-like object b.\n\n        Returns an int representing the number of bytes read (0 for EOF), or","sourceCodeStart":607,"sourceCodeEnd":643,"githubUrl":"https://github.com/RustPython/RustPython/blob/aaeab4f754b4f40efc0c8ab39cf7c4a3c35a8cfd/Lib/_pyio.py#L607-L643","documentation":"ValueError raised inside RawIOBase.read(size) (the default read implementation shown at the SOURCE) after it allocates a buffer of exactly size bytes and delegates to the subclass hook readinto(b): the hook returned n that is negative or larger than len(b). This breaks the readinto contract - return the number of bytes actually written into the buffer, 0 <= n <= len(b). It never happens with stdlib streams; it always indicates a buggy custom read()/readinto() in a RawIOBase/IOBase subclass, and the ValueError fires before the corrupted count can poison higher layers.","triggerScenarios":"A custom stream whose readinto returns len(data) it wanted to write rather than the bytes actually placed into b; readinto returning -1 for EOF instead of 0; readinto delegating to another object's readinto on a smaller buffer and forwarding the outer count; test fakes that ignore the buffer length.","commonSituations":"Wrapping sockets, serial ports, or memoryviews in the io interfaces; porting C-style APIs that use negative return codes for errors/EOF; in-memory fake streams in unit tests that copy more than requested.","solutions":["Fix readinto to clamp and report the real write count: n = min(len(data), len(b)); b[:n] = data[:n]; return n","Return 0 (never a negative) at EOF","Add a unit test asserting 0 <= f.readinto(bytearray(k)) <= k for several buffer sizes k"],"exampleFix":"# before\nclass MemSource(io.RawIOBase):\n    def __init__(self, data):\n        self.data, self.pos = data, 0\n    def readinto(self, b):\n        chunk = self.data[self.pos:self.pos + 4096]   # ignores len(b)\n        b[:] = chunk                                  # fails when len(b) < 4096\n        return len(chunk)\n\n# after\nclass MemSource(io.RawIOBase):\n    def __init__(self, data):\n        self.data, self.pos = data, 0\n    def readinto(self, b):\n        chunk = self.data[self.pos:self.pos + len(b)]\n        b[:len(chunk)] = chunk\n        self.pos += len(chunk)\n        return len(chunk)                             # 0 <= n <= len(b)","handlingStrategy":"try-catch","validationCode":"def check_readinto_contract(source, sizes=(1, 8, 64, 4096)):\n    'Probe a custom readinto before trusting read() on it.'\n    for k in sizes:\n        buf = bytearray(k)\n        n = source.readinto(buf)\n        assert n is None or 0 <= n <= k, f'readinto returned {n} for buffer {k}'","typeGuard":"def is_contract_ok(source) -> bool:\n    try:\n        buf = bytearray(8)\n        n = source.readinto(buf)\n        return n is None or 0 <= n <= 8\n    except Exception:\n        return False","tryCatchPattern":"try:\n    data = raw.read(4096)\nexcept ValueError as e:\n    if 'readinto returned' in str(e):\n        raise RuntimeError(f'custom stream broke the readinto contract: {e}') from e\n    raise","preventionTips":["In custom readinto, slice to len(b) and return the count actually written","Return 0 at EOF, never a negative sentinel","Unit-test readinto with buffers of several sizes before wrapping it in buffered readers"],"tags":["io","readinto","rawiobase","valueerror","custom-stream","contract-violation"],"backgroundTag":"readinto-contract-violation","analyzedSha":"aaeab4f754b4f40efc0c8ab39cf7c4a3c35a8cfd","analyzedAt":"2026-08-17T00:37:52.100Z","schemaVersion":2},"datasetVersion":"2026-08-17T04:17:16.089Z"}