JuliusBrussee/caveman · error · MiddlewareError

runtime_unavailable

runtime_unavailable

Error message

runtime_unavailable

What it means

MiddlewareError raised by _http as a fallback when the middleware returns a non-2xx status and either the body is not JSON, is not a dict, or its error.code is missing/invalid (fails validate.token). It means the server failed, but the client could not extract a more specific error code.

Solutions

1. Check the middleware service is running and reachable (curl -i the endpoint) and that the URL/port points at the middleware, not a proxy UI
2. Inspect middleware logs for the failing request to find the real status/cause
3. Upgrade the SDK and middleware together so error response shapes match the expected {"error":{"code":...}} contract
4. Retry after fixing infrastructure; this error is usually environmental, not a client bug

Example fix

# before (service down)
client.ready()  # MiddlewareError: runtime_unavailable (502 HTML from nginx)
# after
subprocess.run(["systemctl", "start", "caveman-middleware"])  # then retry client.ready()

def middleware_up(endpoint):
    import urllib.request
    try:
        with urllib.request.urlopen(endpoint, timeout=5) as r:
            return r.status < 500
    except Exception:
        return False

def has_valid_error_code(body) -> bool:
    return isinstance(body, dict) and isinstance(body.get("error", {}).get("code"), str) and bool(body["error"]["code"])

for attempt in range(3):
    try:
        return client.ready()
    except MiddlewareError as e:
        if e.args[0] == "runtime_unavailable" and attempt < 2:
            time.sleep(2 ** attempt)
            continue
        raise

When it happens

Trigger: Any of ready()/optimize()/retrieve()/observe()/delete_session() hitting an endpoint returning 4xx/5xx with an unparseable body: HTML error page from a proxy, empty 502 from a crashed gateway, or JSON whose error.code is not a valid token.

Common situations: Middleware process down (502/503 from reverse proxy); wrong port hitting a different service; TLS-terminating proxy emitting HTML error pages; middleware version returning an error shape the SDK doesn't expect.

Understand the failure class

Background: "API error: {status}" and "HTTP 401/403/404/429/5xx" errors: non-2xx HTTP responses explained — this error's family across 27 libraries.

Related errors

• payload_limit
• redirect_refused
• runtime_unavailable
• awscreds: sts assume role with web identity failed
• binary download failed: HTTP

AI-assisted analysis of JuliusBrussee/caveman@3ee70a1026 (2026-09-20). Data as JSON: /api/errors/74082a606168fdf0. Report an issue: GitHub.

Appendix: source

Thrown at packages/sdk/python/caveman_cloud/middleware/runtime.py:402

            bound()
            with connection.getresponse() as response:
                if 300 <= response.status < 400:
                    raise MiddlewareError("redirect_refused")
                content = bytearray()
                while True:
                    bound()
                    # read1 plus the remaining socket deadline bounds slow,
                    # chunked bodies without buffering beyond the response cap.
                    part = response.read1(min(65536, (4 << 20) + 1 - len(content)))
                    if not part:
                        break
                    content.extend(part)
                    if len(content) > 4 << 20:
                        raise MiddlewareError("payload_limit")
                data = json.loads(content.decode("utf-8"))
                if not 200 <= response.status < 300:
                    code = data.get("error", {}).get("code") if isinstance(data, dict) else None
                    raise MiddlewareError(code if validate.token(code) else "runtime_unavailable")
                return data
        finally:
            connection.close()
            with self._lock:
                self._connections.discard(connection)

View on GitHub (pinned to 3ee70a1026)