run-llama/llama_index · error · NotImplementedError

Output parser is not supported for streaming.

Error message

Output parser is not supported for streaming.

What it means

Raised by LLM.stream_predict when either the prompt template or the LLM itself has an output_parser configured. LlamaIndex cannot apply an output parser to a token stream because parsing rules (e.g. regex extraction, Pydantic validation) need the complete text, so it deliberately refuses instead of silently returning unparsed tokens.

Source

Thrown at llama-index-core/llama_index/core/llms/llm.py:691

            ```

        """
        self._log_template_data(prompt, **prompt_args)

        dispatcher.event(
            LLMPredictStartEvent(template=prompt, template_args=prompt_args)
        )
        if self.metadata.is_chat_model:
            messages = self._get_messages(prompt, **prompt_args)
            chat_response = self.stream_chat(messages)
            stream_tokens = stream_chat_response_to_tokens(chat_response)
        else:
            formatted_prompt = self._get_prompt(prompt, **prompt_args)
            stream_response = self.stream_complete(formatted_prompt, formatted=True)
            stream_tokens = stream_completion_response_to_tokens(stream_response)

        if prompt.output_parser is not None or self.output_parser is not None:
            raise NotImplementedError("Output parser is not supported for streaming.")

        return stream_tokens

    @dispatcher.span
    async def apredict(
        self,
        prompt: BasePromptTemplate,
        **prompt_args: Any,
    ) -> str:
        """
        Async Predict for a given prompt.

        Args:
            prompt (BasePromptTemplate):
                The prompt to use for prediction.
            prompt_args (Any):
                Additional arguments to format the prompt with.

View on GitHub (pinned to afd0fef371)

Solutions

  1. Use the non-streaming llm.predict(prompt, **args) instead — output parsers are fully supported there.
  2. Remove the output_parser from the prompt/LLM, consume the stream, concatenate the tokens, then run parser.parse(text) on the final string yourself.
  3. If you need structured output, use llm.structured_predict(OutputCls, prompt) which is designed for that job.

Example fix

// before
const tokens = llm.stream_predict(promptWithParser);
// after (python)
# tokens = llm.predict(prompt_with_parser)  # non-streaming, parser applied
# or parse manually after streaming:
text = "".join(llm.stream_prompt(prompt_without_parser))
result = my_output_parser.parse(text)
Defensive patterns

Strategy: validation

Validate before calling

def can_stream_predict(llm, prompt) -> bool:
    return prompt.output_parser is None and getattr(llm, "output_parser", None) is None

Type guard

def has_no_output_parser(prompt) -> bool:
    return getattr(prompt, "output_parser", None) is None

Try / catch

try:
    tokens = llm.stream_predict(prompt)
except NotImplementedError as e:
    if "Output parser" in str(e):
        text = llm.predict(prompt)
    else:
        raise

Prevention

When it happens

Trigger: Calling llm.stream_predict(prompt) (or a chain that routes through it) where prompt.output_parser is not None (e.g. PromptTemplate(..., output_parser=...)) or where the LLM instance was constructed with an output_parser.

Common situations: Copying a non-streaming predict() example that uses a structured/regex output parser and switching the call to stream_predict; setting Settings.llm to an LLM configured with a global output_parser and then using any streaming prompt API.

Related errors

AI-assisted analysis of run-llama/llama_index@afd0fef371 (2026-08-15). Data as JSON: /api/errors/d2944776cbb2e16a. Report an issue: GitHub.