{"record":{"id":"668edb49282624ac","repo":"opendatalab/MinerU","slug":"pdf-bytes-list-image-writer-list-and-lang-list-m","errorCode":null,"errorMessage":"pdf_bytes_list, image_writer_list, and lang_list must have the same length","messagePattern":"pdf_bytes_list, image_writer_list, and lang_list must have the same length","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"mineru/backend/pipeline/pipeline_analyze.py","lineNumber":168,"sourceCode":"            _finalize_processing_window_context(\n                context,\n                on_doc_ready,\n                client_side_output_generation=client_side_output_generation,\n            )\n\n\ndef doc_analyze_streaming(\n        pdf_bytes_list,\n        image_writer_list,\n        lang_list,\n        on_doc_ready,\n        parse_method: str = 'auto',\n        formula_enable=True,\n        table_enable=True,\n        client_side_output_generation=False,\n):\n    if not (len(pdf_bytes_list) == len(image_writer_list) == len(lang_list)):\n        raise ValueError(\"pdf_bytes_list, image_writer_list, and lang_list must have the same length\")\n\n    doc_contexts = []\n    try:\n        total_pages = 0\n        for doc_index, (pdf_bytes, image_writer, lang) in enumerate(\n            zip(pdf_bytes_list, image_writer_list, lang_list)\n        ):\n            _ocr_enable = _get_ocr_enable(pdf_bytes, parse_method)\n            pdf_doc = open_pdfium_document(pdfium.PdfDocument, pdf_bytes)\n            try:\n                page_count = get_pdfium_document_page_count(pdf_doc)\n                context = {\n                    'doc_index': doc_index,\n                    'pdf_bytes': pdf_bytes,\n                    'pdf_doc': pdf_doc,\n                    'page_count': page_count,\n                    'next_page_idx': 0,\n                    'middle_json': init_middle_json(),","sourceCodeStart":150,"sourceCodeEnd":186,"githubUrl":"https://github.com/opendatalab/MinerU/blob/4fe4bde114a23ee5dd637eae99b767f4669bf58c/mineru/backend/pipeline/pipeline_analyze.py#L150-L186","documentation":"Raised by doc_analyze_streaming() in the pipeline backend as a strict precondition: the three parallel lists pdf_bytes_list, image_writer_list, and lang_list must have identical lengths because they are zipped document-by-document. Any length mismatch aborts before documents are opened, preventing silent partial processing.","triggerScenarios":"Building the three lists from different sources or filters, e.g. pdf bytes gathered per batch but langs defaulted once, or one document dropped from image_writer_list while kept in pdf_bytes_list.","commonSituations":"Dynamic batch construction where one list is appended conditionally; refactors that map over one list but not the others; mixing per-directory and per-file collection logic.","solutions":["Construct all three lists in a single loop so they cannot diverge.","Add an assertion len(...) == len(...) == len(...) at the call site with a descriptive message.","Zip from a single source of truth: docs = [{'bytes':..., 'writer':..., 'lang':...}] then unpack."],"exampleFix":"# before\npdfs = [d.read_bytes() for d in docs if d.size]\nlangs = [\"ch\"] * len(docs)          # docs filtered, langs not\nwriters = make_writers(len(docs))\n\ndoc_analyze_streaming(pdfs, writers, langs, ...)\n\n# after\njobs = [(d.read_bytes(), make_writer(d), d.lang) for d in docs]\ndoc_analyze_streaming([b for b,_,_ in jobs], [w for _,w,_ in jobs], [l for _,_,l in jobs], ...)","handlingStrategy":"validation","validationCode":"def validate_batch(pdf_bytes_list, image_writer_list, lang_list) -> None:\n    n = len(pdf_bytes_list)\n    if not (len(image_writer_list) == n and len(lang_list) == n):\n        raise ValueError(f\"parallel lists must all have length {n}\")","typeGuard":"def is_aligned_batch(pdfs: list, writers: list, langs: list) -> bool:\n    return len(pdfs) == len(writers) == len(langs)","tryCatchPattern":"try:\n    doc_analyze_streaming(pdfs, writers, langs, on_doc_ready)\nexcept ValueError as e:\n    if \"same length\" in str(e):\n        rebuild_lists_from_single_source()  # fix and retry\n    else:\n        raise","preventionTips":["Build the three lists in one loop from a single document-source collection.","Prefer a list of per-document records over parallel arrays.","Assert lengths with a clear message at the call site in debug builds."],"tags":["pipeline","precondition","batching","validation"],"backgroundTag":null,"analyzedSha":"4fe4bde114a23ee5dd637eae99b767f4669bf58c","analyzedAt":"2026-08-14T21:29:18.456Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}