zylon-ai/private-gpt · error · ValueError

No items returned from astream_structured_predict

Error message

No items returned from astream_structured_predict

What it means

ValueError raised by astructured_predict when the async structured stream yielded zero items. Items are only appended when a response chunk has non-empty content AND _parse_partial_json returns a value; zero items means every chunk was empty, whitespace, '{}' (treated as no data), or unparseable.

Source

Thrown at private_gpt/components/llm/custom/structured_mixin.py:177

                yield output

    async def astructured_predict(
        self,
        output_cls: type[Model],
        prompt: PromptTemplate,
        llm_kwargs: dict[str, Any] | None = None,
        **prompt_args: Any,
    ) -> Model:
        items: list[Model | FlexibleModel] = []
        async for item in await self.astream_structured_predict(
            output_cls=output_cls,
            prompt=prompt,
            llm_kwargs=llm_kwargs,
            **prompt_args,
        ):
            items.append(item)
        if not items:
            raise ValueError("No items returned from astream_structured_predict")
        last_item: Model | FlexibleModel = items[-1]
        if isinstance(last_item, FlexibleModel):
            raise ValueError(
                "Last item is a FlexibleModel, expected a specific output_cls."
            )
        return last_item  # type: ignore[return-value]

    async def astream_structured_predict(
        self,
        output_cls: type[Model],
        prompt: PromptTemplate,
        llm_kwargs: dict[str, Any] | None = None,
        **prompt_args: Any,
    ) -> typing.AsyncGenerator[Model | FlexibleModel, None]:
        messages = [
            ChatMessage(
                role=MessageRole.USER,
                content=await asyncio.to_thread(prompt.format, **prompt_args),

View on GitHub (pinned to 4a030776a3)

Solutions

1. Log each raw chunk from astream_structured_predict directly to confirm whether any content arrives at all.
2. If content arrives but never parses, capture it and compare with output_cls (likely markdown-wrapped or truncated JSON).
3. Check backend settings: max_tokens, content filters, and that the model/endpoint actually supports structured output.
4. Retry with a simpler prompt to rule out refusal/empty-completion behavior.

Example fix

# before
result = await llm.astructured_predict(Answer, prompt)

# after
async for item in await llm.astream_structured_predict(Answer, prompt):
    print(item)  # confirm whether any chunks yield items
result = await llm.astructured_predict(Answer, prompt)

try:
    result = await llm.astructured_predict(Answer, prompt)
except ValueError as e:
    if 'No items returned' in str(e):
        logger.warning('empty structured stream; retrying once')
        await asyncio.sleep(1)
        result = await llm.astructured_predict(Answer, prompt)

When it happens

Trigger: Calling astructured_predict where the model/backend returns an empty completion, an immediate refusal, content filtered to nothing, or a stream that errors out mid-flight producing no usable chunks; also output_cls that rejects everything while FlexibleModel fallback also fails (malformed JSON at every step).

Common situations: Empty model responses due to content filters or max_tokens=0 style misconfigurations; backends that return usage-only chunks with empty content; prompts the model refuses; RAG contexts producing blank completions.

Related errors

• No items returned from astream_structured_chat
• No response was generated
• Failed to describe audio in the message.
• Failed to describe images in the message.
• INVALID_REQUEST_ERROR

AI-assisted analysis of zylon-ai/private-gpt@4a030776a3 (2026-08-15). Data as JSON: /api/errors/01edf954fb21ffd2. Report an issue: GitHub.