zylon-ai/private-gpt · error · InvalidFileError

zpgt.ingest.parsing_failure.error

zpgt.ingest.parsing_failure.error

Error message

zpgt.ingest.parsing_failure.error

What it means

InvalidFileError (a CeleryError) raised in IngestionComponent when the parsed document produced more nodes than node_store_component.max_nodes allows. The error carries the IngestionParseErrors.PARSING_FAILURE code (zpgt.ingest.parsing_failure.error) plus the accumulated warnings, even though the file technically parsed — the node-count cap turns success into failure. Logged at info level with both counts before raising.

Source

Thrown at private_gpt/components/ingest/ingest_component.py:205

        ) as notification:
            logger.info("Transforming file into documents: %s", file_info.file_name)

            result: FileParseResult = self.parse_component.file_to_nodes(
                file_info=file_info,
                file_metadata=file_metadata,
                notification=notification,
                warnings=warnings,
            )
            nodes = result.nodes

            max_nodes = self.node_store_component.max_nodes
            if max_nodes and len(nodes) > max_nodes:
                logger.info(
                    "Number of nodes (%d) exceeds the maximum number of nodes (%d)",
                    len(nodes),
                    max_nodes,
                )
                raise InvalidFileError(
                    errors=[IngestionParseErrors.PARSING_FAILURE], warnings=warnings
                )

            for document in nodes:
                # Store artifact and collection metadata
                document.metadata[MetadataKeys.ARTIFACT_ID.value] = artifact
                document.metadata[MetadataKeys.COLLECTION.value] = collection

                # Store LLM and Embedding model metadata
                # to know which models were used to ingest the document
                llm_model = self.llm_component.alias
                if llm_model:
                    document.metadata[MetadataKeys.LLM_MODEL.value] = llm_model
                embed_model = self.embedding_component.get_alias()
                if embed_model:
                    document.metadata[MetadataKeys.EMBED_MODEL.value] = embed_model
                document.metadata.update(file_metadata or {})

View on GitHub (pinned to 4a030776a3)

Solutions

  1. Increase node_store max_nodes in settings to accommodate the document
  2. Reduce node count via coarser chunking (larger chunk_size, less overlap) before ingest
  3. Split the document into smaller files and ingest them separately
  4. If node counts look absurd for the file size, inspect the parsed nodes for a parser/chunking misconfiguration

Example fix

# before: 100k-node document vs max_nodes=10_000 -> InvalidFileError
# after (settings)
# node_store:
#   max_nodes: 200000
Defensive patterns

Strategy: try-catch

Validate before calling

max_nodes = node_store_component.max_nodes
result = parser.parse(file)
if max_nodes and len(result.nodes) > max_nodes:
    raise ValueError(f'document yields {len(result.nodes)} nodes > cap {max_nodes}; split or rechunk')

Try / catch

try:
    ingest_result = component._ingest(file, collection)
except InvalidFileError as e:
    if IngestionParseErrors.PARSING_FAILURE in (e.errors or []):
        return error_response(413, 'document too large: node count exceeds limit', warnings=e.warnings)
    raise

Prevention

When it happens

Trigger: Ingesting a very large document (many pages/sections) where the parser returns len(nodes) > max_nodes, with a nonzero max_nodes configured in the node store component. Also reachable when a parser explodes a small file into a huge number of fragments.

Common situations: Uploading big PDFs or long transcripts with a default node cap; chunking settings (small chunk size / overlap) inflating node counts; a parser bug splitting per character/line; lowering max_nodes to protect memory and then ingesting existing corpora.

Related errors

AI-assisted analysis of zylon-ai/private-gpt@4a030776a3 (2026-08-15). Data as JSON: /api/errors/a41adcaf0e4520f4. Report an issue: GitHub.