JuliusBrussee/caveman · error · MiddlewareError

recovery_unavailable

recovery_unavailable

Error message

recovery_unavailable

What it means

In the Agno adapter's synchronous recover callback, the middleware keeps the current execution frame in a context-local `active` slot. If `recover(handle, ...)` is invoked when no caveman attempt frame is active, the middleware cannot determine which scope/store to retrieve recorded context from, so it raises MiddlewareError('recovery_unavailable').

Solutions

1. Ensure recover() is only called inside a run that entered the middleware (active frame set)
2. Call recover from the same thread/context that started the caveman attempt, or capture the frame before spawning work
3. Wrap external recover calls with an explicit check/logging so failures degrade gracefully
4. Check library version/changelog for threading-context fixes and upgrade

Example fix

# before
result = json.loads(recover(handle, run_context))  # called outside middleware frame
# after
# only call recover inside the middleware-managed run / same context:
with middleware.run(scope="my-scope"):
    result = json.loads(recover(handle, run_context))

frame_ok = middleware_active()  # expose/check ContextVar presence before calling recover
if not frame_ok:
    raise RuntimeError('recover called outside a middleware run')

def can_recover(adapter) -> bool:
    return adapter.active.get() is not None

try:
    payload = recover(handle, run_context)
except MiddlewareError as e:
    if str(e) == 'recovery_unavailable':
        payload = None  # fall back: no active frame
    else:
        raise

When it happens

Trigger: Calling the registered `recover` function outside of a run managed by the Agno middleware, or after the frame was popped (e.g. recover invoked from a different thread/task than the one that entered the attempt).

Common situations: Wiring recover() into a custom retry handler that runs after the middleware frame closed; calling recover from a worker thread or separate coroutine where ContextVar was never set; invoking recover in a plain script without starting a middleware-wrapped run.

Understand the failure class

Background: "Invalid state transition" errors: "status must be X, actually Y", "already rejected/charging/uninstalled", "cannot ... while running" — what they mean when a library rejects your call — this error's family across 31 libraries.

Related errors

• ASGI context must come from authenticated server state
• AutoGen requires a stable Caveman Scope for each agent or…
• Configure exact POST paths and native LLM protocols
• invalid_configuration
• invalid_endpoint

AI-assisted analysis of JuliusBrussee/caveman@3ee70a1026 (2026-09-20). Data as JSON: /api/errors/f54503c4f37af3a0. Report an issue: GitHub.

Appendix: source

Thrown at packages/middleware/python/caveman_middleware/agno.py:145

class _Connection:
    def __init__(self, runtime, scope):
        self.sync = runtime if isinstance(runtime, MiddlewareRuntime) else None
        self.async_runtime = runtime.as_async() if self.sync else runtime
        self.scope, self.recovery_tool = scope, None
        self.active = contextvars.ContextVar("caveman_agno_run", default=None)
        self.version_supported = matches_framework(("agno", "3.0", "4"))
        if not self.version_supported and runtime.mode != "off":
            runtime.decline("unsupported_version")

    def passive(self, runtime, reason):
        return Attempt(runtime, None, str(uuid.uuid4()), str(uuid.uuid4()), passive=True, reason=reason, adapter=ADAPTER.id), {}, None, None

    def register(self):
        def recover(handle: str, run_context: RunContext, offset: int = 0, limit: int = 262144, query: str = ""):
            frame = self.active.get()
            if frame is None:
                raise MiddlewareError("recovery_unavailable")
            if self.sync is None:
                raise TypeError("Synchronous Agno recovery requires MiddlewareRuntime")
            if run_context is not None:
                raise_if_cancelled(run_context.run_id)
            return json.dumps(self.sync.retrieve(frame.scope, handle=handle, offset=offset, limit=limit, query=query),
                              ensure_ascii=False, separators=(",", ":"))

        async def arecover(handle: str, run_context: RunContext, offset: int = 0, limit: int = 262144, query: str = ""):
            frame = self.active.get()
            if frame is None:
                raise MiddlewareError("recovery_unavailable")
            if run_context is not None:
                await araise_if_cancelled(run_context.run_id)
            return json.dumps(await self.async_runtime.retrieve(frame.scope, handle=handle, offset=offset, limit=limit, query=query),
                              ensure_ascii=False, separators=(",", ":"))

        # Agno preserves a Function's entrypoint identity in its per-run copy.
        # Explicit processing avoids schema rewriting and keeps RunContext hidden.

View on GitHub (pinned to 3ee70a1026)