docling-project/docling · error · SystemExit

Dataset '{args.repo_id}' does not contain a 'pdf/' subfolder

Error message

Dataset '{args.repo_id}' does not contain a 'pdf/' subfolder.

What it means

SystemExit raised by _resolve_input_dir() in perfs/iterate_pdf_pages.py after successfully downloading a Hugging Face dataset snapshot: the expected 'pdf/' subfolder is missing from the downloaded snapshot. The script filters the snapshot with allow_patterns='pdf/**', so if the dataset has no pdf/ directory the download yields no usable PDFs and the script aborts.

Source

Thrown at perfs/iterate_pdf_pages.py:422

        from huggingface_hub import snapshot_download
    except ImportError as exc:
        raise SystemExit(
            "Hugging Face dataset support requires 'huggingface_hub' to be installed."
        ) from exc

    assert args.repo_id is not None
    with _suppress_huggingface_output():
        snapshot_dir = Path(
            snapshot_download(
                repo_id=args.repo_id,
                repo_type="dataset",
                revision=args.revision,
                allow_patterns="pdf/**",
            )
        )
    pdf_dir = snapshot_dir / "pdf"
    if not pdf_dir.is_dir():
        raise SystemExit(
            f"Dataset '{args.repo_id}' does not contain a 'pdf/' subfolder."
        )
    return pdf_dir


def _metrics_file_for_thread_count(
    metrics_file: Path | None,
    thread_count: int,
    multiple_thread_counts: bool,
) -> Path | None:
    if metrics_file is None or not multiple_thread_counts:
        return metrics_file
    return metrics_file.with_name(
        f"{metrics_file.stem}-threads-{thread_count}{metrics_file.suffix}"
    )


def _format_duration(seconds: float) -> str:

View on GitHub (pinned to 61d76f1ff3)

Solutions

1. Check the dataset layout on the Hub (Files tab) and confirm PDFs live under 'pdf/'.
2. If they live elsewhere, clone/download the dataset yourself and pass --input-dir <local_dir_with_pdfs> instead of --repo-id.
3. If you own the dataset, restructure it so PDFs are under pdf/ (the pattern the script hardcodes).

Example fix

# before
$ python perfs/iterate_pdf_pages.py --repo-id my-org/corpus
# SystemExit: Dataset 'my-org/corpus' does not contain a 'pdf/' subfolder.

# after
$ hf download my-org/corpus --repo-type dataset --local-dir ./corpus
$ python perfs/iterate_pdf_pages.py --input-dir ./corpus/documents

from pathlib import Path
local = Path("corpus")
if (local / "pdf").is_dir():
    args.input_dir = local / "pdf"
else:
    raise SystemExit(f"{local} lacks pdf/ subfolder; check dataset layout")

When it happens

Trigger: Running with --repo-id pointing at a dataset whose files live under a different layout (e.g. 'documents/', 'data/', or PDFs at the repository root) instead of the required 'pdf/' folder.

Common situations: Pointing the benchmark at a private/internal corpus that follows a different directory convention than the public dataset the script was built against; dataset reorganizations or renames of the pdf/ folder; repo_type mistakes (passing a model repo id instead of a dataset id).

Related errors

• Hugging Face dataset support requires 'huggingface_hub' to b
• Input directory does not exist: {input_dir}
• {input_file} is not valid JSONL or a JSON summary report.
• {input_file} does not look like an iterate_pdf_pages.py summ
• No metrics files found in summary report: {input_file}

AI-assisted analysis of docling-project/docling@61d76f1ff3 (2026-08-14). Data as JSON: /api/errors/6927027e3ea3b998. Report an issue: GitHub.