{"record":{"id":"8e428d1a74b8a1f5","repo":"JuliusBrussee/caveman","slug":"synchronous-litellm-calls-require-middlewareruntime","errorCode":null,"errorMessage":"Synchronous LiteLLM calls require MiddlewareRuntime","messagePattern":"Synchronous LiteLLM calls require MiddlewareRuntime","errorType":"validation","errorClass":"TypeError","httpStatus":null,"severity":"error","filePath":"packages/middleware/python/caveman_middleware/litellm.py","lineNumber":302,"sourceCode":"            owner.reset(owner_token)\n            _active.reset(active_token)\n\n    def _sync(self, method, scope, kwargs):\n        function = getattr(self.client, method)\n        if owner.get() is not None:\n            return function(**kwargs)\n        passive_reason = self._passive_reason(method, kwargs)\n        if passive_reason:\n            attempt = Attempt(self.runtime, None, str(uuid.uuid4()), str(uuid.uuid4()), passive=True,\n                              reason=passive_reason, adapter=\"litellm\")\n            attempt.observe(\"dispatch_intent\")\n            token = owner.set(attempt)\n            try:\n                return function(**kwargs)\n            finally:\n                owner.reset(token)\n        if not isinstance(self.runtime, MiddlewareRuntime):\n            raise TypeError(\"Synchronous LiteLLM calls require MiddlewareRuntime\")\n        router = isinstance(self.client, native.Router) and method == \"completion\"\n        if router and self.runtime.strict:\n            # Logging callbacks swallow their exceptions. Reject this capability\n            # before calling the Router so strict failure cannot dispatch a request.\n            self._report(\"unsupported_sync_router_strict\")\n            raise MiddlewareError(\"unsupported_sync_router_strict\")\n        if router and self._registrations == 0:\n            self._report(\"not_registered\")\n            raise RuntimeError(\"Use CavemanLiteLLM as a context manager while sync Router calls and streams are active\")\n        with self._activation(scope, kwargs, method) as (params, request):\n            if router:\n                token = _sync_router.set(self)\n                try:\n                    return function(**params)\n                finally:\n                    _sync_router.reset(token)\n            # Direct sync SDK calls already name the selected provider/model.\n            session = self._session(request, method, self.runtime) if self.client is native else None","sourceCodeStart":284,"sourceCodeEnd":320,"githubUrl":"https://github.com/JuliusBrussee/caveman/blob/3ee70a102609e550bd2e68004bf5990a9341c851/packages/middleware/python/caveman_middleware/litellm.py#L284-L320","documentation":"Synchronous LiteLLM calls (completion, responses) execute on the caller's thread and need the full MiddlewareRuntime surface (strict mode, sync recovery, reporting). _sync checks isinstance(self.runtime, MiddlewareRuntime) and raises TypeError if a lighter/async-only runtime object was supplied.","triggerScenarios":"Constructing CavemanLiteLLM with an async-only runtime (e.g. the result of runtime.as_async() or a custom object) and then invoking the sync completion()/responses() path.","commonSituations":"Sharing one adapter configured for an async app (FastAPI) inside sync scripts or Celery workers; passing a mock runtime in tests that isn't a MiddlewareRuntime subclass.","solutions":["Construct the adapter with a full MiddlewareRuntime instance for sync usage","Create a separate adapter with a sync-capable runtime instead of reusing the async-configured one","If you are in async code, call acompletion/aresponses instead of the sync methods","In tests, use the real MiddlewareRuntime (or a subclass) rather than arbitrary mocks"],"exampleFix":"// before\nasync_runtime = runtime.as_async()\nadapter = CavemanLiteLLM(runtime=async_runtime)\nadapter.completion(scope=scope, ...)  # TypeError\n\n// after\nadapter = CavemanLiteLLM(runtime=runtime)\nadapter.completion(scope=scope, ...)","handlingStrategy":"type-guard","validationCode":"from caveman_cloud.middleware import MiddlewareRuntime\nif not isinstance(runtime, MiddlewareRuntime):\n    raise TypeError(\"sync LiteLLM calls require a MiddlewareRuntime instance\")","typeGuard":"def supports_sync_calls(runtime) -> bool:\n    from caveman_cloud.middleware import MiddlewareRuntime\n    return isinstance(runtime, MiddlewareRuntime)","tryCatchPattern":"try:\n    return adapter.completion(scope=scope, **kwargs)\nexcept TypeError as e:\n    if \"require MiddlewareRuntime\" in str(e):\n        adapter = CavemanLiteLLM(runtime=middleware_runtime, client=adapter.client)\n        return adapter.completion(scope=scope, **kwargs)","preventionTips":["Construct adapters with the full MiddlewareRuntime, not runtime.as_async() output, when sync calls are possible","In async-only apps, restrict yourself to acompletion/aresponses","Avoid mock runtimes that don't subclass MiddlewareRuntime in tests","Document per-adapter which runtime types are accepted"],"tags":["python","type-error","sync","runtime","litellm"],"backgroundTag":"type-mismatch","analyzedSha":"3ee70a102609e550bd2e68004bf5990a9341c851","analyzedAt":"2026-09-20T15:53:39.229Z","contentChangedAt":"2026-09-20T15:53:39.229Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}