encode/httpx · error · StreamClosed

Attempted to read or stream content, but the stream has been

Error message

Attempted to read or stream content, but the stream has been closed.

What it means

This is httpx.StreamClosed raised from UnattachedStream.__iter__ (sync). An UnattachedStream is the sentinel object httpx installs on a Request/Response after it has been pickled (see Request.__setstate__ / Response.__setstate__). The original I/O stream cannot survive serialization, so any synchronous iteration attempt on a deserialized object is treated as an invalid stream operation.

Source

Thrown at httpx/_content.py:100

            chunk = await self._stream.aread(self.CHUNK_SIZE)
            while chunk:
                yield chunk
                chunk = await self._stream.aread(self.CHUNK_SIZE)
        else:
            # Otherwise iterate.
            async for part in self._stream:
                yield part


class UnattachedStream(AsyncByteStream, SyncByteStream):
    """
    If a request or response is serialized using pickle, then it is no longer
    attached to a stream for I/O purposes. Any stream operations should result
    in `httpx.StreamClosed`.
    """

    def __iter__(self) -> Iterator[bytes]:
        raise StreamClosed()

    async def __aiter__(self) -> AsyncIterator[bytes]:
        raise StreamClosed()
        yield b""  # pragma: no cover


def encode_content(
    content: str | bytes | Iterable[bytes] | AsyncIterable[bytes],
) -> tuple[dict[str, str], SyncByteStream | AsyncByteStream]:
    if isinstance(content, (bytes, str)):
        body = content.encode("utf-8") if isinstance(content, str) else content
        content_length = len(body)
        headers = {"Content-Length": str(content_length)} if body else {}
        return headers, ByteStream(body)

    elif isinstance(content, Iterable) and not isinstance(content, dict):
        # `not isinstance(content, dict)` is a bit oddly specific, but it
        # catches a case that's easy for users to make in error, and would

View on GitHub (pinned to b5addb64f0)

Solutions

1. Read the body fully (`response.read()` / `response.content`) BEFORE serializing the Response, then send the bytes separately.
2. Do not pass Request/Response objects across process boundaries; instead pass primitive data (status, headers dict, content bytes) and reconstruct.
3. If you must pickle, extract `resp.content`, `resp.status_code`, `dict(resp.headers)` and rebuild a Response with content=... on the other side.
4. Use a contextvar/thread-local to keep the live Response in one process and only pass derived data to workers.

Example fix

// before
import pickle, httpx
resp = httpx.get('https://example.com')
blob = pickle.dumps(resp)
r = pickle.loads(blob)
for chunk in r.iter_bytes():  # StreamClosed
    ...
// after
resp = httpx.get('https://example.com')
body = resp.content  # materialize first
payload = {'status': resp.status_code, 'headers': dict(resp.headers), 'body': body}
blob = pickle.dumps(payload)
# in the worker:
p = pickle.loads(blob)
r = httpx.Response(p['status'], headers=p['headers'], content=p['body'])

# Detect an unattached stream before iterating
from httpx._content import UnattachedStream
if isinstance(getattr(obj, 'stream', None), UnattachedStream):
    raise RuntimeError('object was pickled; stream is detached')

from httpx._content import UnattachedStream

def is_stream_live(httpx_obj) -> bool:
    return not isinstance(getattr(httpx_obj, 'stream', None), UnattachedStream)

try:
    for chunk in resp.iter_bytes():
        ...
except httpx.StreamClosed:
    # object was deserialized; fall back to .content if materialized
    body = getattr(resp, '_content', b'')

When it happens

Trigger: Pickle/multiprocessing a Request or Response and then calling .read(), .iter_bytes(), or iterating .stream on the restored object synchronously; using a joblib/ProcessPoolExecutor worker that receives a serialized httpx Response and tries to stream its body; copy.deepcopy on a Response in some configurations.

Common situations: Passing httpx responses across process boundaries (Celery, multiprocessing, dask); caching layers that pickle responses; test fixtures that pickle/unpickle responses for replay; distributed task queues shipping HTTP responses.

Related errors

• Attempted to send an async request with a sync Client instan
• Attempted to send a sync request with an AsyncClient instanc
• Attempted to read or stream some content, but the content ha
• Attempted to access streaming request content, without havin
• Attempted to access streaming response content, without havi

AI-assisted analysis of encode/httpx@b5addb64f0 (2026-08-04). Data as JSON: /data/errors/7090a07c2677d418.json. Report an issue: GitHub.