BerriAI/litellm · error · NotImplementedError

Use AsyncGoogleGenAIGenerateContentStreamingIterator for asy

Error message

Use AsyncGoogleGenAIGenerateContentStreamingIterator for async iteration

What it means

The sync GoogleGenAIGenerateContentStreamingIterator only implements __next__ (backed by _next_google_genai_sse_chunk over a sync SSE stream). Its __anext__ deliberately raises NotImplementedError to stop you from awaiting a sync iterator; async consumption requires AsyncGoogleGenAIGenerateContentStreamingIterator, which wraps an async stream and async logging.

Source

Thrown at litellm/google_genai/streaming_iterator.py:149

    def __iter__(self):
        return self

    def __next__(self):
        try:
            chunk: Final = _next_google_genai_sse_chunk(self.stream_iterator)
            self.collected_chunks.append(chunk)
            return chunk
        except StopIteration:
            raise StopIteration

    def __aiter__(self):
        return self

    async def __anext__(self):
        # This should not be used for sync responses
        # If you need async iteration, use AsyncGoogleGenAIGenerateContentStreamingIterator
        raise NotImplementedError("Use AsyncGoogleGenAIGenerateContentStreamingIterator for async iteration")


class AsyncGoogleGenAIGenerateContentStreamingIterator(BaseGoogleGenAIGenerateContentStreamingIterator):
    """
    Async streaming iterator specifically for Google GenAI generate content API.
    """

    def __init__(
        self,
        response,
        model: str,
        logging_obj: LiteLLMLoggingObj,
        generate_content_provider_config: BaseGoogleGenAIGenerateContentConfig,
        litellm_metadata: dict,
        custom_llm_provider: str,
        request_body: dict | None = None,
        hidden_params: dict[str, Any] | None = None,
    ):

View on GitHub (pinned to 6c2dcb801b)

Solutions

Use the async generate_content call so you receive AsyncGoogleGenAIGenerateContentStreamingIterator, then `async for`
If stuck with the sync iterator, consume it with a normal `for` loop (optionally inside loop.run_in_executor / asyncio.to_thread)
Type-check the iterator class before choosing the loop flavor

Example fix

# before
it = generate_content(model=m, contents=c, stream=True)  # sync iterator
async for chunk in it:  # NotImplementedError
    ...

# after
it = await agenerate_content(model=m, contents=c, stream=True)
async for chunk in it:
    ...

Defensive patterns

Strategy: type-guard

Validate before calling

from litellm.google_genai.streaming_iterator import AsyncGoogleGenAIGenerateContentStreamingIterator

if not isinstance(it, AsyncGoogleGenAIGenerateContentStreamingIterator):
    it = await agenerate_content(model=m, contents=c, stream=True)  # get async iterator

Type guard

from litellm.google_genai.streaming_iterator import (
    AsyncGoogleGenAIGenerateContentStreamingIterator,
)

def is_async_genai_iterator(it) -> bool:
    return isinstance(it, AsyncGoogleGenAIGenerateContentStreamingIterator)

Try / catch

try:
    chunk = await it.__anext__()
except NotImplementedError:
    for chunk in it:  # sync consumption fallback
        process(chunk)

Prevention

Match generate_content sync/async flavor to consumer
Consume sync iterators in threads when inside async servers

When it happens

Trigger: Getting a sync streaming iterator from generate_content (stream=True, sync call) and then using `async for chunk in it` or `await it.__anext__()` — e.g. calling sync code inside an async function and trying to consume the result asynchronously.

Common situations: FastAPI handlers calling the sync generate_content for convenience then iterating with `async for`; refactors that made the caller async but kept the sync retrieval call.

Related errors

AI-assisted analysis of BerriAI/litellm@6c2dcb801b (2026-08-15). Data as JSON: /api/errors/63c9b66a863006c8. Report an issue: GitHub.