{"record":{"id":"d2944776cbb2e16a","repo":"run-llama/llama_index","slug":"output-parser-is-not-supported-for-streaming","errorCode":null,"errorMessage":"Output parser is not supported for streaming.","messagePattern":"Output parser is not supported for streaming\\.","errorType":"exception","errorClass":"NotImplementedError","httpStatus":null,"severity":"error","filePath":"llama-index-core/llama_index/core/llms/llm.py","lineNumber":691,"sourceCode":"            ```\n\n        \"\"\"\n        self._log_template_data(prompt, **prompt_args)\n\n        dispatcher.event(\n            LLMPredictStartEvent(template=prompt, template_args=prompt_args)\n        )\n        if self.metadata.is_chat_model:\n            messages = self._get_messages(prompt, **prompt_args)\n            chat_response = self.stream_chat(messages)\n            stream_tokens = stream_chat_response_to_tokens(chat_response)\n        else:\n            formatted_prompt = self._get_prompt(prompt, **prompt_args)\n            stream_response = self.stream_complete(formatted_prompt, formatted=True)\n            stream_tokens = stream_completion_response_to_tokens(stream_response)\n\n        if prompt.output_parser is not None or self.output_parser is not None:\n            raise NotImplementedError(\"Output parser is not supported for streaming.\")\n\n        return stream_tokens\n\n    @dispatcher.span\n    async def apredict(\n        self,\n        prompt: BasePromptTemplate,\n        **prompt_args: Any,\n    ) -> str:\n        \"\"\"\n        Async Predict for a given prompt.\n\n        Args:\n            prompt (BasePromptTemplate):\n                The prompt to use for prediction.\n            prompt_args (Any):\n                Additional arguments to format the prompt with.\n","sourceCodeStart":673,"sourceCodeEnd":709,"githubUrl":"https://github.com/run-llama/llama_index/blob/afd0fef371831f9bda13e5af7167cf4e981278ab/llama-index-core/llama_index/core/llms/llm.py#L673-L709","documentation":"Raised by LLM.stream_predict when either the prompt template or the LLM itself has an output_parser configured. LlamaIndex cannot apply an output parser to a token stream because parsing rules (e.g. regex extraction, Pydantic validation) need the complete text, so it deliberately refuses instead of silently returning unparsed tokens.","triggerScenarios":"Calling llm.stream_predict(prompt) (or a chain that routes through it) where prompt.output_parser is not None (e.g. PromptTemplate(..., output_parser=...)) or where the LLM instance was constructed with an output_parser.","commonSituations":"Copying a non-streaming predict() example that uses a structured/regex output parser and switching the call to stream_predict; setting Settings.llm to an LLM configured with a global output_parser and then using any streaming prompt API.","solutions":["Use the non-streaming llm.predict(prompt, **args) instead — output parsers are fully supported there.","Remove the output_parser from the prompt/LLM, consume the stream, concatenate the tokens, then run parser.parse(text) on the final string yourself.","If you need structured output, use llm.structured_predict(OutputCls, prompt) which is designed for that job."],"exampleFix":"// before\nconst tokens = llm.stream_predict(promptWithParser);\n// after (python)\n# tokens = llm.predict(prompt_with_parser)  # non-streaming, parser applied\n# or parse manually after streaming:\ntext = \"\".join(llm.stream_prompt(prompt_without_parser))\nresult = my_output_parser.parse(text)","handlingStrategy":"validation","validationCode":"def can_stream_predict(llm, prompt) -> bool:\n    return prompt.output_parser is None and getattr(llm, \"output_parser\", None) is None","typeGuard":"def has_no_output_parser(prompt) -> bool:\n    return getattr(prompt, \"output_parser\", None) is None","tryCatchPattern":"try:\n    tokens = llm.stream_predict(prompt)\nexcept NotImplementedError as e:\n    if \"Output parser\" in str(e):\n        text = llm.predict(prompt)\n    else:\n        raise","preventionTips":["Keep output parsers off LLM instances you intend to stream with.","Before streaming, check prompt.output_parser is None and llm.output_parser is None.","Reserve predict()/structured_predict() for parser-backed prompts."],"tags":["llama-index","streaming","output-parser","not-implemented"],"backgroundTag":null,"analyzedSha":"afd0fef371831f9bda13e5af7167cf4e981278ab","analyzedAt":"2026-08-15T05:42:58.429Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}