aio-libs/aiohttp · error · TooManyMembersError

Compressed stream has more than {MAX_DECOMPRESS_MEMBERS} mem

Error message

Compressed stream has more than {MAX_DECOMPRESS_MEMBERS} members

What it means

Raised by the concat-stream decompressor (used for zstd and multi-member brotli) when a single compressed response contains more than MAX_DECOMPRESS_MEMBERS concatenated members. The cap exists because an attacker can pack a huge number of tiny members into one body to burn CPU/time on decompressor re-initialization. Breaching the limit aborts with TooManyMembersError before the resource abuse can continue.

Source

Thrown at aiohttp/compression_utils.py:255

    @abstractmethod
    def _new_decompressor(self) -> _DecompressObjT:
        """Return a decompressor for the next member."""

    def _decompress_members(self, first: bytes, max_length: int) -> bytes:
        """Decode the members following the one ``first`` came from."""
        remaining = memoryview(self._decompressor.unused_data)
        parts = [first]
        produced = len(first)
        pos = 0
        window = MEMBER_WINDOW_MIN
        budget = max_length
        members = 1

        while pos < len(remaining):
            if self._decompressor.eof:
                members += 1
                if members > MAX_DECOMPRESS_MEMBERS:
                    raise TooManyMembersError(
                        f"Compressed stream has more than "
                        f"{MAX_DECOMPRESS_MEMBERS} members"
                    )
                # Replace the spent decompressor before the budget check below
                # can break out of the loop: it still lists these bytes in its
                # unused_data and would hand them back on the next call.
                self._decompressor = self._new_decompressor()
                window = MEMBER_WINDOW_MIN
            if max_length != self._unlimited:
                budget = max_length - produced
                if budget <= 0:
                    self._pending_unused_data = bytes(remaining[pos:])
                    break

            end = min(pos + window, len(remaining))
            chunk = self._decompressor.decompress(remaining[pos:end], budget)
            if chunk:
                parts.append(chunk)

View on GitHub (pinned to d041d4d0fd)

Solutions

Catch TooManyMembersError and treat the response as malformed (do not retry the same body).
Restrict Accept-Encoding to identity or gzip when the upstream is untrusted.
Cap client_max_size / response body size so abusive payloads are rejected earlier.
Report the endpoint to the operator if it is a legitimate partner feeding malformed encodings.

Example fix

# before
resp = await session.get(url)
data = await resp.read()
# after
from aiohttp.compression_utils import TooManyMembersError
try:
    data = await resp.read()
except TooManyMembersError:
    resp.close()
    raise ClientPayloadError('too many compressed members')

Defensive patterns

Strategy: try-catch

Validate before calling

null

Type guard

null

Try / catch

from aiohttp.compression_utils import TooManyMembersError
try:
    body = await resp.read()
except TooManyMembersError:
    resp.close()
    log.warning('rejected multi-member stream from %s', resp.url)

Prevention

Treat TooManyMembersError as malformed-content, not a transient failure.
For untrusted upstreams, send Accept-Encoding: gzip, identity to avoid br/zstd concat streams.
Cap response body size at the session/server level to reject abusive payloads early.

When it happens

Trigger: A server (or MITM) returns a Content-Encoding: zstd or br body that is many concatenated streams; a corrupted/truncated upload re-assembled into one body; a zip-bomb-style payload crafted against the decompressor.

Common situations: Misconfigured origin that fragments responses; malicious endpoint during scraping; proxy that re-encodes and concatenates; bug in upstream encoder producing thousands of frames.

Related errors

AI-assisted analysis of aio-libs/aiohttp@d041d4d0fd (2026-08-11). Data as JSON: /api/errors/ba92b76deebfa4d9. Report an issue: GitHub.