opendatalab/MinerU · error · ValueError

pdf_bytes_list, image_writer_list, and lang_list must have t

Error message

pdf_bytes_list, image_writer_list, and lang_list must have the same length

What it means

Raised by doc_analyze_streaming() in the pipeline backend as a strict precondition: the three parallel lists pdf_bytes_list, image_writer_list, and lang_list must have identical lengths because they are zipped document-by-document. Any length mismatch aborts before documents are opened, preventing silent partial processing.

Source

Thrown at mineru/backend/pipeline/pipeline_analyze.py:168

            _finalize_processing_window_context(
                context,
                on_doc_ready,
                client_side_output_generation=client_side_output_generation,
            )


def doc_analyze_streaming(
        pdf_bytes_list,
        image_writer_list,
        lang_list,
        on_doc_ready,
        parse_method: str = 'auto',
        formula_enable=True,
        table_enable=True,
        client_side_output_generation=False,
):
    if not (len(pdf_bytes_list) == len(image_writer_list) == len(lang_list)):
        raise ValueError("pdf_bytes_list, image_writer_list, and lang_list must have the same length")

    doc_contexts = []
    try:
        total_pages = 0
        for doc_index, (pdf_bytes, image_writer, lang) in enumerate(
            zip(pdf_bytes_list, image_writer_list, lang_list)
        ):
            _ocr_enable = _get_ocr_enable(pdf_bytes, parse_method)
            pdf_doc = open_pdfium_document(pdfium.PdfDocument, pdf_bytes)
            try:
                page_count = get_pdfium_document_page_count(pdf_doc)
                context = {
                    'doc_index': doc_index,
                    'pdf_bytes': pdf_bytes,
                    'pdf_doc': pdf_doc,
                    'page_count': page_count,
                    'next_page_idx': 0,
                    'middle_json': init_middle_json(),

View on GitHub (pinned to 4fe4bde114)

Solutions

Construct all three lists in a single loop so they cannot diverge.
Add an assertion len(...) == len(...) == len(...) at the call site with a descriptive message.
Zip from a single source of truth: docs = [{'bytes':..., 'writer':..., 'lang':...}] then unpack.

Example fix

# before
pdfs = [d.read_bytes() for d in docs if d.size]
langs = ["ch"] * len(docs)          # docs filtered, langs not
writers = make_writers(len(docs))

doc_analyze_streaming(pdfs, writers, langs, ...)

# after
jobs = [(d.read_bytes(), make_writer(d), d.lang) for d in docs]
doc_analyze_streaming([b for b,_,_ in jobs], [w for _,w,_ in jobs], [l for _,_,l in jobs], ...)

Defensive patterns

Strategy: validation

Validate before calling

def validate_batch(pdf_bytes_list, image_writer_list, lang_list) -> None:
    n = len(pdf_bytes_list)
    if not (len(image_writer_list) == n and len(lang_list) == n):
        raise ValueError(f"parallel lists must all have length {n}")

Type guard

def is_aligned_batch(pdfs: list, writers: list, langs: list) -> bool:
    return len(pdfs) == len(writers) == len(langs)

Try / catch

try:
    doc_analyze_streaming(pdfs, writers, langs, on_doc_ready)
except ValueError as e:
    if "same length" in str(e):
        rebuild_lists_from_single_source()  # fix and retry
    else:
        raise

Prevention

Build the three lists in one loop from a single document-source collection.
Prefer a list of per-document records over parallel arrays.
Assert lengths with a clear message at the call site in debug builds.

When it happens

Trigger: Building the three lists from different sources or filters, e.g. pdf bytes gathered per batch but langs defaulted once, or one document dropped from image_writer_list while kept in pdf_bytes_list.

Common situations: Dynamic batch construction where one list is appended conditionally; refactors that map over one list but not the others; mixing per-directory and per-file collection logic.

Related errors

AI-assisted analysis of opendatalab/MinerU@4fe4bde114 (2026-08-14). Data as JSON: /api/errors/668edb49282624ac. Report an issue: GitHub.