{"record":{"id":"f528f59070459d05","repo":"unclecode/crawl4ai","slug":"invalid-scanline-structure","errorCode":null,"errorMessage":"Invalid scanline structure","messagePattern":"Invalid scanline structure","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"warning","filePath":"crawl4ai/processors/pdf/utils.py","lineNumber":13,"sourceCode":"import re\n\ndef apply_png_predictor(data, width, bits, color_channels):\n    \"\"\"Decode PNG predictor (PDF 1.5+ filter)\"\"\"\n    bytes_per_pixel = (bits * color_channels) // 8\n    if (bits * color_channels) % 8 != 0:\n        bytes_per_pixel += 1\n        \n    stride = width * bytes_per_pixel\n    scanline_length = stride + 1  # +1 for filter byte\n    \n    if len(data) % scanline_length != 0:\n        raise ValueError(\"Invalid scanline structure\")\n    \n    num_lines = len(data) // scanline_length\n    output = bytearray()\n    prev_line = b'\\x00' * stride\n    \n    for i in range(num_lines):\n        line = data[i*scanline_length:(i+1)*scanline_length]\n        filter_type = line[0]\n        filtered = line[1:]\n        \n        if filter_type == 0:  # None\n            decoded = filtered\n        elif filter_type == 1:  # Sub\n            decoded = bytearray(filtered)\n            for j in range(bytes_per_pixel, len(decoded)):\n                decoded[j] = (decoded[j] + decoded[j - bytes_per_pixel]) % 256\n        elif filter_type == 2:  # Up\n            decoded = bytearray([(filtered[j] + prev_line[j]) % 256 ","sourceCodeStart":1,"sourceCodeEnd":31,"githubUrl":"https://github.com/unclecode/crawl4ai/blob/7e801521428ee12509994d39151006f64055ebe3/crawl4ai/processors/pdf/utils.py#L1-L31","documentation":"ValueError from apply_png_predictor, the pure-Python decoder for PNG-predictor FlateDecode streams in PDF 1.5+ images. It validates that the raw byte stream divides evenly into scanlines of (stride + 1) bytes (one filter byte plus pixel data); a remainder means the data does not match the declared width/bits/channels geometry and cannot be decoded.","triggerScenarios":"The PDF image XObject's /Width, /BitsPerComponent, or /ColorChannels-derived value disagrees with the actual decoded byte length — corrupt PDFs, malformed encoders, or a caller invoking apply_png_predictor directly with mismatched parameters.","commonSituations":"Crawling third-party/scanned PDFs from broken generators, OCR tools emitting non-standard predictors, or truncated downloads where the Flate stream was cut short. Rare with well-formed PDFs.","solutions":["Treat it as a data problem first: re-download the PDF (truncation is a common cause) and verify it opens in a standard viewer.","Confirm the width/bits/color_channels arguments match the image dictionary (/Width, /BitsPerComponent, /ColorSpace size) if you call this utility yourself.","Wrap image extraction per-image/per-page so one malformed image doesn't abort the whole document — text extraction can still proceed.","If the PDF is genuinely malformed, skip image extraction for that document (extract_images=False)."],"exampleFix":"# before\nraw = zlib.decompress(stream_data)\npixels = apply_png_predictor(raw, width, bits, color_channels)  # ValueError: Invalid scanline structure\n\n# after\nraw = zlib.decompress(stream_data)\ntry:\n    pixels = apply_png_predictor(raw, width, bits, color_channels)\nexcept ValueError:\n    logger.warning(f\"Skipping malformed image on page {page_num}\")\n    continue","handlingStrategy":"try-catch","validationCode":"def predictor_stream_valid(data: bytes, width: int, bits: int, color_channels: int) -> bool:\n    bpp = (bits * color_channels) // 8 + (1 if (bits * color_channels) % 8 else 0)\n    scanline = width * bpp + 1\n    return len(data) > 0 and len(data) % scanline == 0","typeGuard":null,"tryCatchPattern":"try:\n    pixels = apply_png_predictor(raw, width, bits, color_channels)\nexcept ValueError as e:\n    logger.warning(f\"Malformed image stream ({e}); skipping\")\n    continue  # per-image recovery, keep processing the document","preventionTips":["Re-download suspicious PDFs before assuming corruption (truncation mimics corruption).","Cross-check /Width, /BitsPerComponent, /ColorSpace against your decoder arguments.","Scope try/except to per-image, not per-document, so one bad image doesn't kill text extraction."],"tags":["pdf","image-extraction","data-corruption","decoding"],"backgroundTag":null,"analyzedSha":"7e801521428ee12509994d39151006f64055ebe3","analyzedAt":"2026-08-14T20:46:20.673Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}