zylon-ai/private-gpt · error · ValueError

No items returned from astream_structured_chat

Error message

No items returned from astream_structured_chat

What it means

Raised by the non-streaming wrapper around astream_structured_chat when the async generator yielded zero items before completing. The wrapper must return the last item of the stream; an empty stream means no chat response (not even a FlexibleModel fallback) was ever produced, which indicates the underlying LLM call failed silently or produced nothing parseable.

Source

Thrown at private_gpt/components/llm/custom/structured_mixin.py:223

        self,
        output_cls: type[Model],
        messages: Sequence[ChatMessage],
        tools: Sequence[BaseTool] | None = None,
        reasoning_effort: ReasoningEffort = ReasoningEffort.NONE,
        allow_flexible: bool = False,
        **kwargs: Any,
    ) -> "Model | FlexibleModel":
        items: list[Model | FlexibleModel] = []
        async for item in await self.astream_structured_chat(
            output_cls=output_cls,
            messages=messages,
            tools=tools,
            reasoning_effort=reasoning_effort,
            **kwargs,
        ):
            items.append(item)
        if not items:
            raise ValueError("No items returned from astream_structured_chat")
        last_item: Model | FlexibleModel = items[-1]
        if isinstance(last_item, FlexibleModel) and not allow_flexible:
            raise ValueError(
                "Last item is a FlexibleModel, expected a specific output_cls."
            )
        return last_item

    async def astream_structured_chat(
        self,
        output_cls: type[Model],
        messages: Sequence[ChatMessage],
        tools: Sequence[BaseTool] | None = None,
        reasoning_effort: ReasoningEffort = ReasoningEffort.NONE,
        **kwargs: Any,
    ) -> typing.AsyncGenerator[Model | FlexibleModel, None]:
        from partial_json_parser.core.options import Allow

        from private_gpt.components.llm.custom.base import StructuredOutputsParams

View on GitHub (pinned to 4a030776a3)

Solutions

1. Retry the call — empty streams are often transient backend/streaming failures.
2. Check the LLM backend logs/health: verify the model responded with actual content for the exact messages sent.
3. Log the raw messages and llm_kwargs to confirm the request is well-formed (non-empty prompt, valid tool definitions).
4. If using a proxy or OpenAI-compatible server, confirm it fully supports the chat/structured endpoint being used.

for attempt in range(3):
    try:
        result = await llm.structured_chat(MyModel, messages=messages)
        break
    except ValueError as e:
        if 'No items returned' not in str(e) or attempt == 2:
            raise

When it happens

Trigger: Calling the structured chat wrapper (structured_chat or the consume-all helper around astream_structured_chat) when the underlying stream ends without yielding: empty completion from the backend, immediate stream termination, cancellation, or an exception path in the backend that closes the generator without emitting items.

Common situations: Misconfigured endpoint returning an empty response; overly aggressive content filters blocking all output; context window exceeded so the model emits nothing; transient backend failures during streaming.

Related errors

• No items returned from astream_structured_predict
• No response was generated
• Failed to describe audio in the message.
• Failed to describe images in the message.
• INVALID_REQUEST_ERROR

AI-assisted analysis of zylon-ai/private-gpt@4a030776a3 (2026-08-15). Data as JSON: /api/errors/84147f25fcce3bed. Report an issue: GitHub.