{"record":{"id":"01edf954fb21ffd2","repo":"zylon-ai/private-gpt","slug":"no-items-returned-from-astream-structured-predict","errorCode":null,"errorMessage":"No items returned from astream_structured_predict","messagePattern":"No items returned from astream_structured_predict","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"private_gpt/components/llm/custom/structured_mixin.py","lineNumber":177,"sourceCode":"                yield output\n\n    async def astructured_predict(\n        self,\n        output_cls: type[Model],\n        prompt: PromptTemplate,\n        llm_kwargs: dict[str, Any] | None = None,\n        **prompt_args: Any,\n    ) -> Model:\n        items: list[Model | FlexibleModel] = []\n        async for item in await self.astream_structured_predict(\n            output_cls=output_cls,\n            prompt=prompt,\n            llm_kwargs=llm_kwargs,\n            **prompt_args,\n        ):\n            items.append(item)\n        if not items:\n            raise ValueError(\"No items returned from astream_structured_predict\")\n        last_item: Model | FlexibleModel = items[-1]\n        if isinstance(last_item, FlexibleModel):\n            raise ValueError(\n                \"Last item is a FlexibleModel, expected a specific output_cls.\"\n            )\n        return last_item  # type: ignore[return-value]\n\n    async def astream_structured_predict(\n        self,\n        output_cls: type[Model],\n        prompt: PromptTemplate,\n        llm_kwargs: dict[str, Any] | None = None,\n        **prompt_args: Any,\n    ) -> typing.AsyncGenerator[Model | FlexibleModel, None]:\n        messages = [\n            ChatMessage(\n                role=MessageRole.USER,\n                content=await asyncio.to_thread(prompt.format, **prompt_args),","sourceCodeStart":159,"sourceCodeEnd":195,"githubUrl":"https://github.com/zylon-ai/private-gpt/blob/4a030776a31a901ad80b1bf4d7faa2c1a367efbb/private_gpt/components/llm/custom/structured_mixin.py#L159-L195","documentation":"ValueError raised by astructured_predict when the async structured stream yielded zero items. Items are only appended when a response chunk has non-empty content AND _parse_partial_json returns a value; zero items means every chunk was empty, whitespace, '{}' (treated as no data), or unparseable.","triggerScenarios":"Calling astructured_predict where the model/backend returns an empty completion, an immediate refusal, content filtered to nothing, or a stream that errors out mid-flight producing no usable chunks; also output_cls that rejects everything while FlexibleModel fallback also fails (malformed JSON at every step).","commonSituations":"Empty model responses due to content filters or max_tokens=0 style misconfigurations; backends that return usage-only chunks with empty content; prompts the model refuses; RAG contexts producing blank completions.","solutions":["Log each raw chunk from astream_structured_predict directly to confirm whether any content arrives at all.","If content arrives but never parses, capture it and compare with output_cls (likely markdown-wrapped or truncated JSON).","Check backend settings: max_tokens, content filters, and that the model/endpoint actually supports structured output.","Retry with a simpler prompt to rule out refusal/empty-completion behavior."],"exampleFix":"# before\nresult = await llm.astructured_predict(Answer, prompt)\n\n# after\nasync for item in await llm.astream_structured_predict(Answer, prompt):\n    print(item)  # confirm whether any chunks yield items\nresult = await llm.astructured_predict(Answer, prompt)","handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"try:\n    result = await llm.astructured_predict(Answer, prompt)\nexcept ValueError as e:\n    if 'No items returned' in str(e):\n        logger.warning('empty structured stream; retrying once')\n        await asyncio.sleep(1)\n        result = await llm.astructured_predict(Answer, prompt)","preventionTips":["Empty streams are usually transient (backend/filter hiccups) — one retry with backoff is cheap insurance.","Log raw chunks from astream_structured_predict when it happens to distinguish empty completions from parse failures."],"tags":["llm","structured-output","async","empty-response","streaming"],"backgroundTag":null,"analyzedSha":"4a030776a31a901ad80b1bf4d7faa2c1a367efbb","analyzedAt":"2026-08-15T03:51:26.951Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}