docling-project/docling · error · SystemExit

{input_file} is not valid JSONL or a JSON summary report.

Error message

{input_file} is not valid JSONL or a JSON summary report.

What it means

SystemExit raised by _resolve_metrics_files() in perfs/plot_memory_metrics.py when the input file is neither a .jsonl metrics file (dispatched earlier by suffix) nor parseable as JSON. The script falls back to treating the input as an iterate_pdf_pages.py JSON summary report, and json.loads() raising JSONDecodeError triggers this combined 'not JSONL nor JSON' message.

Source

Thrown at perfs/plot_memory_metrics.py:40

    )
    parser.add_argument(
        "-o",
        "--output",
        type=Path,
        default=Path("memory-metrics.png"),
        help="Output plot path.",
    )
    return parser.parse_args()


def _resolve_metrics_files(input_file: Path) -> list[tuple[str, Path]]:
    if input_file.suffix.lower() == ".jsonl":
        return [(input_file.stem, input_file)]

    try:
        report = json.loads(input_file.read_text(encoding="utf-8"))
    except json.JSONDecodeError as exc:
        raise SystemExit(
            f"{input_file} is not valid JSONL or a JSON summary report."
        ) from exc

    runs = report.get("runs")
    if not isinstance(runs, list):
        raise SystemExit(
            f"{input_file} does not look like an iterate_pdf_pages.py summary report."
        )

    metrics_files: list[tuple[str, Path]] = []
    for index, run in enumerate(runs, start=1):
        if not isinstance(run, dict):
            continue
        metrics_file = run.get("metrics_file")
        if not isinstance(metrics_file, str):
            continue
        thread_count = run.get("threads")
        label = (

View on GitHub (pinned to 61d76f1ff3)

Solutions

1. If the file is line-delimited JSON events, rename it to end with .jsonl so the suffix fast path handles it.
2. If it should be a summary report, validate it parses: 'python -m json.tool report.json' and regenerate it by re-running iterate_pdf_pages.py.
3. Check the file tail for truncation ('tail -c 200 file') and re-run the benchmark if the writer was killed mid-report.

Example fix

# before
$ python perfs/plot_memory_metrics.py runs/metrics_events.txt   # jsonl content, wrong suffix
# SystemExit: ... is not valid JSONL or a JSON summary report.

# after
$ mv runs/metrics_events.txt runs/metrics_events.jsonl
$ python perfs/plot_memory_metrics.py runs/metrics_events.jsonl

import json
from pathlib import Path

p = Path(input_file)
if p.suffix.lower() != ".jsonl":
    try:
        json.loads(p.read_text(encoding="utf-8"))
    except json.JSONDecodeError:
        raise SystemExit(f"{p} is neither .jsonl nor valid JSON; regenerate it")

When it happens

Trigger: Passing a --input file whose suffix is not .jsonl (so the JSONL fast path is skipped) and whose content is invalid JSON — e.g. a partially written report, a JSONL file renamed to .json, or a log file.

Common situations: A benchmark run was interrupted while writing the summary report, leaving truncated JSON; concatenating or hand-editing report files; passing the raw .jsonl metrics path but with a non-.jsonl extension, defeating the suffix check.

Related errors

• {input_file} does not look like an iterate_pdf_pages.py summ
• No metrics files found in summary report: {input_file}
• Hugging Face dataset support requires 'huggingface_hub' to b
• Dataset '{args.repo_id}' does not contain a 'pdf/' subfolder
• Input directory does not exist: {input_dir}

AI-assisted analysis of docling-project/docling@61d76f1ff3 (2026-08-14). Data as JSON: /api/errors/d5a309d1c353daad. Report an issue: GitHub.