JuliusBrussee/caveman · error · ValueError

Recovery executor belongs to another runtime or scope

Error message

Recovery executor belongs to another runtime or scope

What it means

When operator_recovery supplies a recovery binding, _remember asks the async runtime whether it owns that binding for this scope (owns_binding). If not, the binding came from another MiddlewareRuntime instance or a different scope, and the adapter raises ValueError rather than mixing recovery executors across runtimes.

Solutions

1. Re-obtain recovery bindings from the same MiddlewareRuntime that is handling the call
2. Rebuild the operator_recovery callback so it uses the current runtime's recovery API instead of cached bindings
3. In dev/test, clear cached bindings on runtime recreation
4. Confirm only one runtime instance is constructed per app lifecycle

Example fix

// before
old_binding = cached_bindings[scope.key]  # from another runtime
adapter = CavemanLiteLLM(runtime=runtime_b, operator_recovery=lambda s: old_binding)

// after
adapter = CavemanLiteLLM(runtime=runtime_b, operator_recovery=runtime_b.make_recovery_binding)

if supplied is not None and not async_runtime.owns_binding(supplied_binding, scope):
    supplied_binding = None  # drop stale binding; let the runtime mint a fresh one

def binding_owned(binding, scope, runtime) -> bool:
    try:
        return bool(runtime.owns_binding(binding, scope))
    except Exception:
        return False

try:
    with adapter.activation(scope, kwargs, method):
        ...
except ValueError as e:
    if "another runtime or scope" in str(e):
        refresh_recovery_bindings(runtime)  # rebuild from the current runtime
        retry_once()

When it happens

Trigger: An operator_recovery callback returns a RecoveryBinding produced by a different MiddlewareRuntime instance, or cached from a previous app run / different scope, and the current async_runtime.owns_binding(binding, scope) returns False.

Common situations: Creating a second runtime (e.g. after reload or in tests) while reusing a pickled/cached recovery binding; wiring one shared recovery executor across multiple runtimes; copying bindings between environments.

Understand the failure class

Background: "Must be a positive integer", "Invalid value", "Unsupported": the invalid-argument-value error family, when a library rejects the value you pass — this error's family across 35 libraries.

Related errors

• Install caveman-middleware[litellm] to use the LiteLLM…
• LiteLLM metadata must be a native dictionary
• LiteLLM scope must be a trusted Caveman Scope
• not_registered
• Saved state must match the configured workbench count

AI-assisted analysis of JuliusBrussee/caveman@3ee70a1026 (2026-09-20). Data as JSON: /api/errors/821915626f9a8d9d. Report an issue: GitHub.

Appendix: source

Thrown at packages/middleware/python/caveman_middleware/litellm.py:101

    def close(self):
        with _registration_lock:
            self._registrations = 1
            self.__exit__()
        with self._lock:
            self._requests.clear()
            self._attempts.clear()

    def _remember(self, scope, call_type=None):
        if not isinstance(scope, Scope):
            raise TypeError("LiteLLM scope must be a trusted Caveman Scope")
        request = _Request(scope, str(uuid.uuid4()), time.monotonic() + 3600, protocol=self._protocol(call_type))
        if self.operator_recovery:
            supplied = self.operator_recovery(scope)
            if supplied is not None:
                request.binding, request.overhead = supplied
                if not self.async_runtime.owns_binding(request.binding, scope):
                    raise ValueError("Recovery executor belongs to another runtime or scope")
        key = uuid.uuid4().hex
        with self._lock:
            now = time.monotonic()
            for old in list(self._requests):
                if self._requests[old].expires <= now:
                    del self._requests[old]
            while len(self._requests) >= 1024:
                self._requests.popitem(last=False)
            self._requests[key] = request
        return key, request

    def _request(self, kwargs):
        params = kwargs.get("litellm_params")
        params = params if plain(params) else {}
        with self._lock:
            for metadata in (kwargs.get("litellm_metadata"), kwargs.get("metadata"), params.get("litellm_metadata"), params.get("metadata")):
                key = metadata.get(_KEY) if plain(metadata) else None
                request = self._requests.get(key) if type(key) is str else None

View on GitHub (pinned to 3ee70a1026)