docling-project/docling · error · SystemExit

{input_file} does not look like an iterate_pdf_pages.py summ

Error message

{input_file} does not look like an iterate_pdf_pages.py summary report.

What it means

SystemExit raised by _resolve_metrics_files() in perfs/plot_memory_metrics.py when the input parses as JSON but does not contain a top-level 'runs' key holding a list. The plotter expects the summary-report schema written by iterate_pdf_pages.py, whose 'runs' array carries per-run entries with 'metrics_file' paths; a JSON file without that structure is rejected.

Source

Thrown at perfs/plot_memory_metrics.py:46

        help="Output plot path.",
    )
    return parser.parse_args()


def _resolve_metrics_files(input_file: Path) -> list[tuple[str, Path]]:
    if input_file.suffix.lower() == ".jsonl":
        return [(input_file.stem, input_file)]

    try:
        report = json.loads(input_file.read_text(encoding="utf-8"))
    except json.JSONDecodeError as exc:
        raise SystemExit(
            f"{input_file} is not valid JSONL or a JSON summary report."
        ) from exc

    runs = report.get("runs")
    if not isinstance(runs, list):
        raise SystemExit(
            f"{input_file} does not look like an iterate_pdf_pages.py summary report."
        )

    metrics_files: list[tuple[str, Path]] = []
    for index, run in enumerate(runs, start=1):
        if not isinstance(run, dict):
            continue
        metrics_file = run.get("metrics_file")
        if not isinstance(metrics_file, str):
            continue
        thread_count = run.get("threads")
        label = (
            f"threads={thread_count}"
            if isinstance(thread_count, int)
            else f"run {index}"
        )
        metrics_path = Path(metrics_file)
        if not metrics_path.is_absolute():

View on GitHub (pinned to 61d76f1ff3)

Solutions

1. Regenerate the summary with the same (current) version of iterate_pdf_pages.py, which writes the runs array, and pass that file to the plotter.
2. Confirm the top-level shape: 'python -c "import json;d=json.load(open('r.json'));print(list(d))"' should show 'runs'.
3. If you only have raw .jsonl metrics, pass the .jsonl file directly instead of wrapping it in JSON.

Example fix

# before
$ python perfs/plot_memory_metrics.py per_run_metrics.json  # a metrics event file
# SystemExit: ... does not look like an iterate_pdf_pages.py summary report.

# after
$ python perfs/iterate_pdf_pages.py --report-file summary.json ...
$ python perfs/plot_memory_metrics.py summary.json

import json

def is_summary_report(path: str) -> bool:
    try:
        data = json.loads(Path(path).read_text(encoding="utf-8"))
    except json.JSONDecodeError:
        return False
    return isinstance(data, dict) and isinstance(data.get("runs"), list)

def is_iterate_summary(data: object) -> bool:
    return isinstance(data, dict) and isinstance(data.get("runs"), list)

When it happens

Trigger: Passing a valid JSON file that is not an iterate_pdf_pages.py summary — e.g. a DoclingDocument export, an experiment config, or a report from an older/newer script version where the key is missing or renamed.

Common situations: Schema drift between script versions (older summaries without 'runs'); passing the per-run metrics JSON file itself (which has event objects, not a runs list) instead of the summary; hand-written JSON wrappers around benchmark output.

Related errors

• No metrics files found in summary report: {input_file}
• {input_file} is not valid JSONL or a JSON summary report.
• Hugging Face dataset support requires 'huggingface_hub' to b
• Dataset '{args.repo_id}' does not contain a 'pdf/' subfolder
• Input directory does not exist: {input_dir}

AI-assisted analysis of docling-project/docling@61d76f1ff3 (2026-08-14). Data as JSON: /api/errors/b859b04363571e12. Report an issue: GitHub.