RustPython/RustPython · error · RuntimeError

unable to write; sendfile is in progress

Error message

unable to write; sendfile is in progress

What it means

While a sendfile transfer is in progress, the proactor write transport installs an internal _empty_waiter future and rejects all other writes with RuntimeError('unable to write; sendfile is in progress'). Interleaving buffered writes with the zero-copy file transfer would corrupt the outgoing byte stream order, so the transport enforces exclusivity until sendfile completes and the waiter resets.

Source

Thrown at Lib/asyncio/proactor_events.py:346

class _ProactorBaseWritePipeTransport(_ProactorBasePipeTransport,
                                      transports.WriteTransport):
    """Transport for write pipes."""

    _start_tls_compatible = True

    def __init__(self, *args, **kw):
        super().__init__(*args, **kw)
        self._empty_waiter = None

    def write(self, data):
        if not isinstance(data, (bytes, bytearray, memoryview)):
            raise TypeError(
                f"data argument must be a bytes-like object, "
                f"not {type(data).__name__}")
        if self._eof_written:
            raise RuntimeError('write_eof() already called')
        if self._empty_waiter is not None:
            raise RuntimeError('unable to write; sendfile is in progress')

        if not data:
            return

        if self._conn_lost:
            if self._conn_lost >= constants.LOG_THRESHOLD_FOR_CONNLOST_WRITES:
                logger.warning('socket.send() raised exception.')
            self._conn_lost += 1
            return

        # Observable states:
        # 1. IDLE: _write_fut and _buffer both None
        # 2. WRITING: _write_fut set; _buffer None
        # 3. BACKED UP: _write_fut set; _buffer a bytearray
        # We always copy the data, so the caller can't modify it
        # while we're still waiting for the I/O to happen.
        if self._write_fut is None:  # IDLE -> WRITING
            assert self._buffer is None

View on GitHub (pinned to aaeab4f754)

Solutions

1. Serialize operations: await the sendfile to completion before issuing any further write() on that transport
2. Gate periodic writers (heartbeats, metrics) with an asyncio.Event so they pause for the duration of the sendfile
3. If you truly need interleaved output, abandon sendfile and stream the file in chunks through regular write()

Example fix

# before
asyncio.create_task(loop.sock_sendfile(sock, big_file))  # fire-and-forget
transport.write(b"next request")                          # RuntimeError

# after
await loop.sock_sendfile(sock, big_file)                  # transfer completes first
transport.write(b"next request")                          # safe now

class ConnectionGate:
    def __init__(self):
        self.sending = asyncio.Event()  # set while sendfile is active
    async def sendfile(self, loop, sock, file):
        self.sending.set()
        try:
            await loop.sock_sendfile(sock, file)
        finally:
            self.sending.clear()
    def can_write(self) -> bool:
        return not self.sending.is_set()

When it happens

Trigger: A heartbeat/keep-alive task calling transport.write() on a connection while await loop.sock_sendfile(sock, file) is still running on it; issuing write() from a callback that fires during the sendfile window; firing sendfile as a background task and immediately writing the next request.

Common situations: HTTP servers streaming large static files with sendfile while periodic ping/metrics writers share the same connection; progress-report tasks racing a download; pipelined request handlers that assume writes queue transparently.

Related errors

• Empty waiter is already set
• data argument must be a bytes-like object, not {type(data)._
• Expected a list of types, an ellipsis, ParamSpec, or Concate
• Expected Theme object, found {t}
• write_eof() already called

AI-assisted analysis of RustPython/RustPython@aaeab4f754 (2026-08-17). Data as JSON: /api/errors/53b1fcf5e7bc3b66. Report an issue: GitHub.