unclecode/crawl4ai · warning · ValueError

Invalid scanline structure

Error message

Invalid scanline structure

What it means

ValueError from apply_png_predictor, the pure-Python decoder for PNG-predictor FlateDecode streams in PDF 1.5+ images. It validates that the raw byte stream divides evenly into scanlines of (stride + 1) bytes (one filter byte plus pixel data); a remainder means the data does not match the declared width/bits/channels geometry and cannot be decoded.

Source

Thrown at crawl4ai/processors/pdf/utils.py:13

import re

def apply_png_predictor(data, width, bits, color_channels):
    """Decode PNG predictor (PDF 1.5+ filter)"""
    bytes_per_pixel = (bits * color_channels) // 8
    if (bits * color_channels) % 8 != 0:
        bytes_per_pixel += 1
        
    stride = width * bytes_per_pixel
    scanline_length = stride + 1  # +1 for filter byte
    
    if len(data) % scanline_length != 0:
        raise ValueError("Invalid scanline structure")
    
    num_lines = len(data) // scanline_length
    output = bytearray()
    prev_line = b'\x00' * stride
    
    for i in range(num_lines):
        line = data[i*scanline_length:(i+1)*scanline_length]
        filter_type = line[0]
        filtered = line[1:]
        
        if filter_type == 0:  # None
            decoded = filtered
        elif filter_type == 1:  # Sub
            decoded = bytearray(filtered)
            for j in range(bytes_per_pixel, len(decoded)):
                decoded[j] = (decoded[j] + decoded[j - bytes_per_pixel]) % 256
        elif filter_type == 2:  # Up
            decoded = bytearray([(filtered[j] + prev_line[j]) % 256 

View on GitHub (pinned to 7e80152142)

Solutions

1. Treat it as a data problem first: re-download the PDF (truncation is a common cause) and verify it opens in a standard viewer.
2. Confirm the width/bits/color_channels arguments match the image dictionary (/Width, /BitsPerComponent, /ColorSpace size) if you call this utility yourself.
3. Wrap image extraction per-image/per-page so one malformed image doesn't abort the whole document — text extraction can still proceed.
4. If the PDF is genuinely malformed, skip image extraction for that document (extract_images=False).

Example fix

# before
raw = zlib.decompress(stream_data)
pixels = apply_png_predictor(raw, width, bits, color_channels)  # ValueError: Invalid scanline structure

# after
raw = zlib.decompress(stream_data)
try:
    pixels = apply_png_predictor(raw, width, bits, color_channels)
except ValueError:
    logger.warning(f"Skipping malformed image on page {page_num}")
    continue

def predictor_stream_valid(data: bytes, width: int, bits: int, color_channels: int) -> bool:
    bpp = (bits * color_channels) // 8 + (1 if (bits * color_channels) % 8 else 0)
    scanline = width * bpp + 1
    return len(data) > 0 and len(data) % scanline == 0

try:
    pixels = apply_png_predictor(raw, width, bits, color_channels)
except ValueError as e:
    logger.warning(f"Malformed image stream ({e}); skipping")
    continue  # per-image recovery, keep processing the document

When it happens

Trigger: The PDF image XObject's /Width, /BitsPerComponent, or /ColorChannels-derived value disagrees with the actual decoded byte length — corrupt PDFs, malformed encoders, or a caller invoking apply_png_predictor directly with mismatched parameters.

Common situations: Crawling third-party/scanned PDFs from broken generators, OCR tools emitting non-standard predictors, or truncated downloads where the Flate stream was cut short. Rare with well-formed PDFs.

Related errors

• Unsupported filter type: {filter_type}
• Timeout downloading PDF from {url}: {str(e)}
• Failed to download PDF from {url}: {str(e)}
• pypdf is required for PDF processing. Install with 'pip inst

AI-assisted analysis of unclecode/crawl4ai@7e80152142 (2026-08-14). Data as JSON: /api/errors/f528f59070459d05. Report an issue: GitHub.