{"record":{"id":"a41adcaf0e4520f4","repo":"zylon-ai/private-gpt","slug":"zpgt-ingest-parsing-failure-error","errorCode":"zpgt.ingest.parsing_failure.error","errorMessage":"zpgt.ingest.parsing_failure.error","messagePattern":"zpgt\\.ingest\\.parsing_failure\\.error","errorType":"error_code","errorClass":"InvalidFileError","httpStatus":400,"severity":"error","filePath":"private_gpt/components/ingest/ingest_component.py","lineNumber":205,"sourceCode":"        ) as notification:\n            logger.info(\"Transforming file into documents: %s\", file_info.file_name)\n\n            result: FileParseResult = self.parse_component.file_to_nodes(\n                file_info=file_info,\n                file_metadata=file_metadata,\n                notification=notification,\n                warnings=warnings,\n            )\n            nodes = result.nodes\n\n            max_nodes = self.node_store_component.max_nodes\n            if max_nodes and len(nodes) > max_nodes:\n                logger.info(\n                    \"Number of nodes (%d) exceeds the maximum number of nodes (%d)\",\n                    len(nodes),\n                    max_nodes,\n                )\n                raise InvalidFileError(\n                    errors=[IngestionParseErrors.PARSING_FAILURE], warnings=warnings\n                )\n\n            for document in nodes:\n                # Store artifact and collection metadata\n                document.metadata[MetadataKeys.ARTIFACT_ID.value] = artifact\n                document.metadata[MetadataKeys.COLLECTION.value] = collection\n\n                # Store LLM and Embedding model metadata\n                # to know which models were used to ingest the document\n                llm_model = self.llm_component.alias\n                if llm_model:\n                    document.metadata[MetadataKeys.LLM_MODEL.value] = llm_model\n                embed_model = self.embedding_component.get_alias()\n                if embed_model:\n                    document.metadata[MetadataKeys.EMBED_MODEL.value] = embed_model\n                document.metadata.update(file_metadata or {})\n","sourceCodeStart":187,"sourceCodeEnd":223,"githubUrl":"https://github.com/zylon-ai/private-gpt/blob/4a030776a31a901ad80b1bf4d7faa2c1a367efbb/private_gpt/components/ingest/ingest_component.py#L187-L223","documentation":"InvalidFileError (a CeleryError) raised in IngestionComponent when the parsed document produced more nodes than node_store_component.max_nodes allows. The error carries the IngestionParseErrors.PARSING_FAILURE code (zpgt.ingest.parsing_failure.error) plus the accumulated warnings, even though the file technically parsed — the node-count cap turns success into failure. Logged at info level with both counts before raising.","triggerScenarios":"Ingesting a very large document (many pages/sections) where the parser returns len(nodes) > max_nodes, with a nonzero max_nodes configured in the node store component. Also reachable when a parser explodes a small file into a huge number of fragments.","commonSituations":"Uploading big PDFs or long transcripts with a default node cap; chunking settings (small chunk size / overlap) inflating node counts; a parser bug splitting per character/line; lowering max_nodes to protect memory and then ingesting existing corpora.","solutions":["Increase node_store max_nodes in settings to accommodate the document","Reduce node count via coarser chunking (larger chunk_size, less overlap) before ingest","Split the document into smaller files and ingest them separately","If node counts look absurd for the file size, inspect the parsed nodes for a parser/chunking misconfiguration"],"exampleFix":"# before: 100k-node document vs max_nodes=10_000 -> InvalidFileError\n# after (settings)\n# node_store:\n#   max_nodes: 200000","handlingStrategy":"try-catch","validationCode":"max_nodes = node_store_component.max_nodes\nresult = parser.parse(file)\nif max_nodes and len(result.nodes) > max_nodes:\n    raise ValueError(f'document yields {len(result.nodes)} nodes > cap {max_nodes}; split or rechunk')","typeGuard":null,"tryCatchPattern":"try:\n    ingest_result = component._ingest(file, collection)\nexcept InvalidFileError as e:\n    if IngestionParseErrors.PARSING_FAILURE in (e.errors or []):\n        return error_response(413, 'document too large: node count exceeds limit', warnings=e.warnings)\n    raise","preventionTips":["Pre-check page/node counts before ingest when users upload large files","Set max_nodes deliberately high enough for your largest real document","Tune chunk size/overlap to keep node counts proportional to content"],"tags":["ingest","chunking","limits","parsing"],"backgroundTag":null,"analyzedSha":"4a030776a31a901ad80b1bf4d7faa2c1a367efbb","analyzedAt":"2026-08-15T03:51:26.951Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}