{"record":{"id":"678127f5b9b0c0d0","repo":"zylon-ai/private-gpt","slug":"last-item-is-a-flexiblemodel-expected-a-specific","errorCode":null,"errorMessage":"Last item is a FlexibleModel, expected a specific output_cls.","messagePattern":"Last item is a FlexibleModel, expected a specific output_cls\\.","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"private_gpt/components/llm/custom/structured_mixin.py","lineNumber":76,"sourceCode":"\n    def structured_predict(\n        self,\n        output_cls: type[Model],\n        prompt: PromptTemplate,\n        llm_kwargs: dict[str, Any] | None = None,\n        **prompt_args: Any,\n    ) -> Model:\n        items: list[Model | FlexibleModel] = list(\n            self.stream_structured_predict(\n                output_cls=output_cls,\n                prompt=prompt,\n                llm_kwargs=llm_kwargs,\n                **prompt_args,\n            )\n        )\n        last_item: Model | FlexibleModel = items[-1]\n        if isinstance(last_item, FlexibleModel):\n            raise ValueError(\n                \"Last item is a FlexibleModel, expected a specific output_cls.\"\n            )\n        return last_item  # type: ignore[return-value]\n\n    def stream_structured_predict(\n        self,\n        output_cls: type[Model],\n        prompt: PromptTemplate,\n        llm_kwargs: dict[str, Any] | None = None,\n        **prompt_args: Any,\n    ) -> Generator[Model | FlexibleModel, None, None]:\n        messages = [\n            ChatMessage(\n                role=MessageRole.USER,\n                content=prompt.format(**prompt_args),\n            )\n        ]\n        return self.stream_structured_chat(","sourceCodeStart":58,"sourceCodeEnd":94,"githubUrl":"https://github.com/zylon-ai/private-gpt/blob/4a030776a31a901ad80b1bf4d7faa2c1a367efbb/private_gpt/components/llm/custom/structured_mixin.py#L58-L94","documentation":"ValueError raised by StructuredChatMixin.structured_predict: it consumes the full stream_structured_predict output and, if the last parsed item is a FlexibleModel (the lenient fallback model used when the streamed JSON could not validate against output_cls), it refuses to return it. This signals the model's final output never conformed to the requested Pydantic schema.","triggerScenarios":"Calling structured_predict where the LLM's accumulated JSON fails output_cls validation at every stream step, so only FlexibleModel instances are emitted — malformed JSON, wrong field names/types, or the model adding prose around the JSON.","commonSituations":"Weak models ignoring schema instructions; schemas with strict constraints (required fields, enums, formats) the model violates; responses wrapped in markdown fences or text that breaks parsing; truncated streams cutting required fields.","solutions":["Inspect the raw completion (log stream chunks) to see what JSON the model actually produced versus output_cls.","Simplify the output schema: fewer required fields, more permissive types, clear field descriptions.","Strengthen the prompt to demand raw JSON only (no prose/markdown fences).","Switch to a model with native structured-output support, or use structured_chat with allow_flexible=True and validate/repair manually."],"exampleFix":"# before\nclass Answer(BaseModel):\n    confidence: float  # model keeps omitting\n\nresult = llm.structured_predict(Answer, prompt)\n\n# after\nclass Answer(BaseModel):\n    confidence: float | None = None\n\nresult = llm.structured_predict(Answer, prompt)","handlingStrategy":"fallback","validationCode":null,"typeGuard":null,"tryCatchPattern":"try:\n    result = llm.structured_predict(Answer, prompt)\nexcept ValueError as e:\n    if 'FlexibleModel' in str(e):\n        raw = llm.structured_chat(Answer, messages, allow_flexible=True)\n        result = repair_and_validate(raw, Answer)  # coerce fields, retry once","preventionTips":["Design schemas the target model can actually satisfy (optional fields, lenient types).","Keep a repair/retry path: capture the flexible output, fix it up, validate against output_cls programmatically."],"tags":["llm","structured-output","json","parsing","validation"],"backgroundTag":null,"analyzedSha":"4a030776a31a901ad80b1bf4d7faa2c1a367efbb","analyzedAt":"2026-08-15T03:51:26.951Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}