{"record":{"id":"05b551f60e42ea05","repo":"zylon-ai/private-gpt","slug":"iterator-next-operation-failed-e-s","errorCode":null,"errorMessage":"Iterator next() operation failed: {e!s}","messagePattern":"Iterator next\\(\\) operation failed: (.+?)","errorType":"exception","errorClass":"AsyncIteratorError","httpStatus":null,"severity":"error","filePath":"private_gpt/utils/async_utils.py","lineNumber":68,"sourceCode":"            # Process items in chunks for better performance\n            chunk = []\n            for _ in range(chunk_size):\n                try:\n\n                    def safe_next(it: Iterator[T]) -> T:\n                        try:\n                            return next(it)\n                        except StopIteration:\n                            return None  # type: ignore\n\n                    item = await loop.run_in_executor(\n                        internal_executor, safe_next, iterator\n                    )\n                    if item is None:  # Handle StopIteration gracefully\n                        break\n                    chunk.append(item)\n                except Exception as e:\n                    raise AsyncIteratorError(\n                        f\"Iterator next() operation failed: {e!s}\"\n                    ) from e\n\n            if not chunk:\n                break\n\n            # Process the chunk\n            for item in chunk:\n                try:\n                    if transform_fn:\n                        # Run transform in executor if it's CPU-intensive\n                        result = await loop.run_in_executor(\n                            internal_executor, transform_fn, item\n                        )\n                        yield result\n                    else:\n                        yield item\n                except Exception as e:","sourceCodeStart":50,"sourceCodeEnd":86,"githubUrl":"https://github.com/zylon-ai/private-gpt/blob/4a030776a31a901ad80b1bf4d7faa2c1a367efbb/private_gpt/utils/async_utils.py#L50-L86","documentation":"Wrapped as AsyncIteratorError by to_async_iterator when calling next() on the underlying synchronous iterator raises any exception other than StopIteration. The next() call runs in a thread-pool executor via loop.run_in_executor; any error raised by the source iterator's __next__ (I/O error, parsing error, or a bug in the generator) propagates and is re-raised with this message, chaining the original cause.","triggerScenarios":"Passing a generator that reads files or hits the network and raises mid-iteration; passing an already-exhausted/closed generator whose next() raises RuntimeError (generator raised StopIteration internally, PEP 479); an iterator over a DB cursor whose connection dropped; a map/filter chain whose underlying callable throws on a specific item.","commonSituations":"Streaming ingestion from a source that fails partway (S3 object deleted, socket reset); iterators built over LangChain document loaders that raise on malformed documents; reusing a generator after it was closed; transform pipelines where an earlier exception surfaces only when the item is pulled.","solutions":["Inspect the chained original exception (raise ... from e) - fix the root cause in the source iterator, not the wrapper","Make the source iterator defensive: catch expected errors inside it and skip/log bad items instead of raising","Ensure generators never let StopIteration escape from inside (PEP 479) - catch it and return explicitly","If iterating a shared/closable resource, verify it is still open before yielding each item"],"exampleFix":"# before\ndef docs():\n    for f in paths:\n        yield parse(f.read())  # raises if f missing -> AsyncIteratorError\n\n# after\ndef docs():\n    for f in paths:\n        try:\n            yield parse(f.read())\n        except OSError as e:\n            logger.warning(f\"skipping {f}: {e}\")\n            continue","handlingStrategy":"try-catch","validationCode":"null","typeGuard":"null","tryCatchPattern":"from private_gpt.utils.async_utils import AsyncIteratorError\n\ntry:\n    async for item in to_async_iterator(iter(docs)):\n        process(item)\nexcept AsyncIteratorError as e:\n    root = e.__cause__  # the real next() failure\n    logger.error(\"source iterator failed: %s\", root, exc_info=root)","preventionTips":["Make the source iterator defensive: catch expected errors internally and skip/log bad items","Never let StopIteration escape from inside a generator (PEP 479) - return explicitly","Log the failing element inside the iterator before raising so failures are reproducible","For network-backed iterators, retry transient errors inside __next__ instead of propagating"],"tags":["async","iterator","streaming","error-wrapping"],"backgroundTag":null,"analyzedSha":"4a030776a31a901ad80b1bf4d7faa2c1a367efbb","analyzedAt":"2026-08-15T03:51:26.951Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}