JuliusBrussee/caveman · error · MiddlewareError

payload_limit

payload_limit

Error message

payload_limit

What it means

MiddlewareError raised by _http when the response body exceeds the hard 4 MiB cap (4 << 20 bytes). The transport streams with read1 and aborts as soon as buffered content would exceed the limit, to prevent unbounded memory use from a misbehaving or hostile server.

Solutions

1. Reduce request size: narrow the optimize/retrieve scope or paginate results so each response stays under 4 MiB
2. If you control the middleware, lower per-request result size or raise the client cap (4 MiB constant in runtime.py) consciously
3. Check the server for loops/bugs that emit unbounded or non-JSON bodies
4. Capture the response with curl to confirm actual body size before adjusting limits

Example fix

// before
result = client.retrieve(handle)  # 6 MiB page -> payload_limit
// after
result = client.retrieve(handle, page_offset=0)  # paginate, each page < 4 MiB

# estimate response size beforehand when the API exposes counts
# e.g. skip retrieve if projected page size exceeds 4 MiB
if estimated_bytes > 4 * 1024 * 1024:
    use_pagination = True

def fits_payload_cap(n_bytes: int, cap: int = 4 << 20) -> bool:
    return 0 < n_bytes <= cap

try:
    data = client.retrieve(handle)
except MiddlewareError as e:
    if e.args[0] == "payload_limit":
        data = fetch_in_pages(handle)  # retry with pagination
    else:
        raise

When it happens

Trigger: Any of ready()/optimize()/retrieve()/observe()/delete_session() receiving a JSON response larger than 4 MiB, or a server that never terminates the body (slow-loris style) until the cap trips.

Common situations: retrieve() on a very large document whose recovery page exceeds 4 MiB; observe() batch payloads too large; a misconfigured or buggy middleware returning an error page/HTML larger than the cap; proxy appending large bodies.

Understand the failure class

Background: payload too large / request exceeds maximum size: why libraries cap bytes and how to fix oversize payloads — this error's family across 50 libraries.

Related errors

• clickhouse response exceeds
• redirect_refused
• runtime_unavailable
• awscreds: sts assume role with web identity failed
• binary download failed: HTTP

AI-assisted analysis of JuliusBrussee/caveman@3ee70a1026 (2026-09-20). Data as JSON: /api/errors/7f5dc63e128c68ea. Report an issue: GitHub.

Appendix: source

Thrown at packages/sdk/python/caveman_cloud/middleware/runtime.py:398

            bound()
            connection.request("POST" if body is not None else "GET", PREFIX + path,
                               body=body.encode("utf-8") if body is not None else None, headers=headers)
            bound()
            with connection.getresponse() as response:
                if 300 <= response.status < 400:
                    raise MiddlewareError("redirect_refused")
                content = bytearray()
                while True:
                    bound()
                    # read1 plus the remaining socket deadline bounds slow,
                    # chunked bodies without buffering beyond the response cap.
                    part = response.read1(min(65536, (4 << 20) + 1 - len(content)))
                    if not part:
                        break
                    content.extend(part)
                    if len(content) > 4 << 20:
                        raise MiddlewareError("payload_limit")
                data = json.loads(content.decode("utf-8"))
                if not 200 <= response.status < 300:
                    code = data.get("error", {}).get("code") if isinstance(data, dict) else None
                    raise MiddlewareError(code if validate.token(code) else "runtime_unavailable")
                return data
        finally:
            connection.close()
            with self._lock:
                self._connections.discard(connection)

View on GitHub (pinned to 3ee70a1026)