{"record":{"id":"52cd29540fb4d191","repo":"microsoft/autogen","slug":"unexpected-response-type-from-llamacpp-model","errorCode":null,"errorMessage":"Unexpected response type from LlamaCpp model.","messagePattern":"Unexpected response type from LlamaCpp model\\.","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/packages/autogen-ext/src/autogen_ext/models/llama_cpp/_llama_cpp_completion_client.py","lineNumber":333,"sourceCode":"\n        if self.model_info[\"function_calling\"]:\n            # Run this in on the event loop to avoid blocking.\n            response_future = asyncio.get_event_loop().run_in_executor(\n                None,\n                lambda: self.llm.create_chat_completion(\n                    messages=converted_messages, tools=convert_tools(tools), stream=False, **create_args\n                ),\n            )\n        else:\n            response_future = asyncio.get_event_loop().run_in_executor(\n                None, lambda: self.llm.create_chat_completion(messages=converted_messages, stream=False, **create_args)\n            )\n        if cancellation_token:\n            cancellation_token.link_future(response_future)\n        response = await response_future\n\n        if not isinstance(response, dict):\n            raise ValueError(\"Unexpected response type from LlamaCpp model.\")\n\n        self._total_usage[\"prompt_tokens\"] += response[\"usage\"][\"prompt_tokens\"]\n        self._total_usage[\"completion_tokens\"] += response[\"usage\"][\"completion_tokens\"]\n\n        # Parse the response\n        response_tool_calls: ChatCompletionTool | None = None\n        response_text: str | None = None\n        if \"choices\" in response and len(response[\"choices\"]) > 0:\n            if \"message\" in response[\"choices\"][0]:\n                response_text = response[\"choices\"][0][\"message\"][\"content\"]\n            if \"tool_calls\" in response[\"choices\"][0]:\n                response_tool_calls = response[\"choices\"][0][\"tool_calls\"]  # type: ignore\n\n        content: List[FunctionCall] | str = \"\"\n        thought: str | None = None\n        if response_tool_calls:\n            content = []\n            for tool_call in response_tool_calls:","sourceCodeStart":315,"sourceCodeEnd":351,"githubUrl":"https://github.com/microsoft/autogen/blob/027ecf0a379bcc1d09956d46d12d44a3ad9cee14/python/packages/autogen-ext/src/autogen_ext/models/llama_cpp/_llama_cpp_completion_client.py#L315-L351","documentation":"After awaiting create_chat_completion (run in an executor), the client requires the result to be a dict, since it immediately indexes response['usage'] and response['choices']. If llama-cpp-python returns something else (e.g. an object, None, or a changed return shape after a monkeypatch or version drift), this ValueError fires. In practice it signals a broken or incompatible llama-cpp-python installation.","triggerScenarios":"llama-cpp-python version whose create_chat_completion returns a non-dict (API drift between the installed wheel and what autogen-ext expects); a mocked/patched Llama object in tests returning MagicMock or a tuple; streaming accidentally enabled inside create_args via extra_create_args (stream=True) so the return value is a generator.","commonSituations":"Pip-installed a pre-release or very old llama-cpp-python; passing extra_create_args={'stream': True} to a non-streaming call; unit tests substituting a fake Llama whose create_chat_completion returns a string; ABI-mismatched wheel returning error objects.","solutions":["Pin a known-good llama-cpp-python version (e.g. pip install 'llama-cpp-python==0.3.x') matching the autogen-ext release's tested range","Remove stream=True from extra_create_args — use create_stream (which itself raises NotImplementedError) or omit it","If testing with a fake Llama, make create_chat_completion return a dict with 'usage' and 'choices' keys"],"exampleFix":"# before\nresult = await client.create(messages, extra_create_args={\"stream\": True})\n\n# after\nresult = await client.create(messages)","handlingStrategy":"fallback","validationCode":"installed = importlib.metadata.version(\"llama-cpp-python\")\nif tuple(int(x) for x in installed.split(\".\")[:2]) < (0, 2):\n    raise RuntimeError(f\"llama-cpp-python {installed} too old; upgrade to a tested release\")","typeGuard":null,"tryCatchPattern":"try:\n    result = await client.create(messages)\nexcept ValueError as e:\n    if \"Unexpected response type\" in str(e):\n        logger.error(\"llama-cpp-python returned a non-dict; check version/patches\")\n        raise\n    raise","preventionTips":["Pin llama-cpp-python to the version range autogen-ext tests against","Never pass stream=True via extra_create_args to create()","In tests, make fake Llama.create_chat_completion return dict responses with usage/choices"],"tags":["llama-cpp","version-compatibility","response-parsing","streaming"],"backgroundTag":null,"analyzedSha":"027ecf0a379bcc1d09956d46d12d44a3ad9cee14","analyzedAt":"2026-08-15T03:38:00.719Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}