zylon-ai/private-gpt · error · ExtractionUnsuccessfulError

Document extraction unsuccessful for '{file_name}': unmapped

Error message

Document extraction unsuccessful for '{file_name}': unmapped-glyph ratio exceeded threshold ({self.config.failure_threshold}).

What it means

Raised as ExtractionUnsuccessfulError by DoclingApiReader.lazy_load_data when _is_extraction_unsuccessful(valid_contents) detects that the ratio of unmapped glyphs exceeds the configured failure_threshold (from the reader's transformation.docling config). It guards against garbage ingestion: PDFs whose fonts fail to map to real text often convert 'successfully' but produce replacement-character junk, which would poison the vector index.

Source

Thrown at private_gpt/components/readers/docling/docling_api_reader.py:183

        try:
            conversion_result = await self.client.convert_from_bytes(
                file_name, file_bytes, to_formats=["md"], pages=pages, **load_kwargs
            )
        except Exception as e:
            raise ValueError(f"Document conversion failed: {e}") from e

        if conversion_result.status not in ["success", "partial_success"]:
            raise ValueError(
                f"Document conversion failed with status: {conversion_result.status}. "
                f"Errors: {conversion_result.errors}"
            )

        contents = self._get_content(conversion_result)
        valid_contents = [content for content in contents if content]
        if not valid_contents:
            raise ValueError("No valid document content found after conversion")
        if self._is_extraction_unsuccessful(valid_contents):
            raise ExtractionUnsuccessfulError(
                f"Document extraction unsuccessful for '{file_name}': unmapped-glyph "
                f"ratio exceeded threshold ({self.config.failure_threshold})."
            )

        docs = [
            self._page_to_doc(
                content=content,
                index=idx,
                include_page_metadata=len(valid_contents) > 1,
                extra_info=extra_info,
            )
            for idx, content in enumerate(valid_contents)
        ]

        if debug_mode:
            logger.info(f"Loaded document from {file_name}")
            logger.info(f"Document has {len(docs)} pages")
            if conversion_result.timings:

View on GitHub (pinned to 4a030776a3)

Solutions

Inspect the extracted text for the file — if it is replacement-character junk, re-OCR the source (enable force_full_page_ocr / use_ocr) or obtain a better source PDF.
If the document's text is acceptable and the guard is too aggressive for your corpus, raise failure_threshold in the docling transformation settings.
Handle ExtractionUnsuccessfulError specifically in the ingestion loop and route the file to manual review/quarantine rather than failing the batch.
For PDFs you control, re-export with embedded fonts to fix the root cause.

Example fix

# ingestion loop — before
nodes = await reader.lazy_load_data(file_info)

# after
from private_gpt.components.readers.docling.docling_api_reader import ExtractionUnsuccessfulError
try:
    nodes = await reader.lazy_load_data(file_info)
except ExtractionUnsuccessfulError:
    quarantine(file_info)  # manual review path
    return []

Defensive patterns

Strategy: try-catch

Validate before calling

def unmapped_glyph_ratio(text: str) -> float:
    if not text:
        return 0.0
    bad = sum(text.count(c) for c in "\ufffd\u25a1")
    return bad / max(len(text), 1)

# pre-check before accepting a converted doc into the index
ratio = unmapped_glyph_ratio(extracted_text)
if ratio > config.failure_threshold:
    route_to_manual_review(file_name)

Try / catch

from private_gpt.components.readers.docling.docling_api_reader import ExtractionUnsuccessfulError

try:
    nodes = [n async for n in reader.lazy_load_data(file_info)]
except ExtractionUnsuccessfulError:
    quarantine(file_info, reason="unmapped-glyph ratio over threshold")
    return []  # keep the batch alive; this file needs a better source or OCR

Prevention

Catch ExtractionUnsuccessfulError separately from ValueError at the ingestion loop.
Tune transformation.docling failure_threshold to your corpus quality instead of disabling the guard.
Prefer fixing the source (re-export with embedded fonts, or full-page OCR) over lowering thresholds indiscriminately.

When it happens

Trigger: Ingesting PDFs with broken/non-embedded fonts (common with old scans, some LaTeX/Windows font subsets, and CAD exports) where conversion succeeds but the text contains a high proportion of unmapped-glyph placeholder characters, pushing the ratio over config.failure_threshold.

Common situations: Batch-ingesting third-party/scanned archives; documents from plotter/CAD tools; threshold too strict after upgrading private-gpt (default sensitivity change); the same documents previously ingested before the guard was added.

Related errors

AI-assisted analysis of zylon-ai/private-gpt@4a030776a3 (2026-08-15). Data as JSON: /api/errors/1b16405bc6592485. Report an issue: GitHub.