{"record":{"id":"63c9b66a863006c8","repo":"BerriAI/litellm","slug":"use-asyncgooglegenaigeneratecontentstreamingiterat","errorCode":null,"errorMessage":"Use AsyncGoogleGenAIGenerateContentStreamingIterator for async iteration","messagePattern":"Use AsyncGoogleGenAIGenerateContentStreamingIterator for async iteration","errorType":"exception","errorClass":"NotImplementedError","httpStatus":null,"severity":"error","filePath":"litellm/google_genai/streaming_iterator.py","lineNumber":149,"sourceCode":"\n    def __iter__(self):\n        return self\n\n    def __next__(self):\n        try:\n            chunk: Final = _next_google_genai_sse_chunk(self.stream_iterator)\n            self.collected_chunks.append(chunk)\n            return chunk\n        except StopIteration:\n            raise StopIteration\n\n    def __aiter__(self):\n        return self\n\n    async def __anext__(self):\n        # This should not be used for sync responses\n        # If you need async iteration, use AsyncGoogleGenAIGenerateContentStreamingIterator\n        raise NotImplementedError(\"Use AsyncGoogleGenAIGenerateContentStreamingIterator for async iteration\")\n\n\nclass AsyncGoogleGenAIGenerateContentStreamingIterator(BaseGoogleGenAIGenerateContentStreamingIterator):\n    \"\"\"\n    Async streaming iterator specifically for Google GenAI generate content API.\n    \"\"\"\n\n    def __init__(\n        self,\n        response,\n        model: str,\n        logging_obj: LiteLLMLoggingObj,\n        generate_content_provider_config: BaseGoogleGenAIGenerateContentConfig,\n        litellm_metadata: dict,\n        custom_llm_provider: str,\n        request_body: dict | None = None,\n        hidden_params: dict[str, Any] | None = None,\n    ):","sourceCodeStart":131,"sourceCodeEnd":167,"githubUrl":"https://github.com/BerriAI/litellm/blob/6c2dcb801bf2b75c18f1bb24140e7cf57465cc4d/litellm/google_genai/streaming_iterator.py#L131-L167","documentation":"The sync GoogleGenAIGenerateContentStreamingIterator only implements __next__ (backed by _next_google_genai_sse_chunk over a sync SSE stream). Its __anext__ deliberately raises NotImplementedError to stop you from awaiting a sync iterator; async consumption requires AsyncGoogleGenAIGenerateContentStreamingIterator, which wraps an async stream and async logging.","triggerScenarios":"Getting a sync streaming iterator from generate_content (stream=True, sync call) and then using `async for chunk in it` or `await it.__anext__()` — e.g. calling sync code inside an async function and trying to consume the result asynchronously.","commonSituations":"FastAPI handlers calling the sync generate_content for convenience then iterating with `async for`; refactors that made the caller async but kept the sync retrieval call.","solutions":["Use the async generate_content call so you receive AsyncGoogleGenAIGenerateContentStreamingIterator, then `async for`","If stuck with the sync iterator, consume it with a normal `for` loop (optionally inside loop.run_in_executor / asyncio.to_thread)","Type-check the iterator class before choosing the loop flavor"],"exampleFix":"# before\nit = generate_content(model=m, contents=c, stream=True)  # sync iterator\nasync for chunk in it:  # NotImplementedError\n    ...\n\n# after\nit = await agenerate_content(model=m, contents=c, stream=True)\nasync for chunk in it:\n    ...","handlingStrategy":"type-guard","validationCode":"from litellm.google_genai.streaming_iterator import AsyncGoogleGenAIGenerateContentStreamingIterator\n\nif not isinstance(it, AsyncGoogleGenAIGenerateContentStreamingIterator):\n    it = await agenerate_content(model=m, contents=c, stream=True)  # get async iterator","typeGuard":"from litellm.google_genai.streaming_iterator import (\n    AsyncGoogleGenAIGenerateContentStreamingIterator,\n)\n\ndef is_async_genai_iterator(it) -> bool:\n    return isinstance(it, AsyncGoogleGenAIGenerateContentStreamingIterator)","tryCatchPattern":"try:\n    chunk = await it.__anext__()\nexcept NotImplementedError:\n    for chunk in it:  # sync consumption fallback\n        process(chunk)","preventionTips":["Match generate_content sync/async flavor to consumer","Consume sync iterators in threads when inside async servers"],"tags":["google-genai","streaming","sync-async-mismatch"],"backgroundTag":null,"analyzedSha":"6c2dcb801bf2b75c18f1bb24140e7cf57465cc4d","analyzedAt":"2026-08-15T07:12:03.035Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}