docling-project/docling · error · EbcdicDecodeError

Input ends inside {name!r}: {end - offset} of {size} bytes l

Error message

Input ends inside {name!r}: {end - offset} of {size} bytes left.

What it means

EbcdicDecodeError raised by the _take helper when the input buffer ends before a prefix field (record length or record type) can be read in full: offset + size exceeds the end of the remaining data. The message names the field and says how many of the expected bytes remain. It indicates truncation or prefix misalignment — either the file was cut short, or record framing drifted so a prefix is being read at a bogus offset.

Source

Thrown at docling/backend/ebcdic_backend.py:182

        record = layout.select(record_type)
        if record is None:
            raise EbcdicDecodeError(
                f"No record layout matches record type {record_type!r}."
            )

        size = record.size if length is None else length - layout.prefix_size
        if size < 0:
            raise EbcdicDecodeError(
                f"Record length {length} is shorter than the "
                f"{layout.prefix_size}-byte record prefix."
            )
        return record, size, offset

    @staticmethod
    def _take(data: bytes, offset: int, size: int, end: int, name: str) -> bytes:
        if offset + size > end:
            raise EbcdicDecodeError(
                f"Input ends inside {name!r}: {end - offset} of {size} bytes left."
            )
        return data[offset : offset + size]

    def _decode_record(self, record: EbcdicRecordLayout, body: bytes) -> list[str]:
        values: list[str] = []
        offset = 0
        for field in record.fields:
            chunk = body[offset : offset + field.size]
            offset += field.size
            if field.type is not EbcdicFieldType.SKIP:
                values.append(str(self._decoder.decode(chunk, field)))
        return values


class EbcdicDocumentBackend(DeclarativeDocumentBackend):
    """Declarative backend converting EBCDIC data files to `DoclingDocument`.

View on GitHub (pinned to 61d76f1ff3)

Solutions

1. Verify the file size against the expected record structure (count x size for fixed-length records) and re-transfer the file in binary mode.
2. If a length field misdecode is the root cause, fix the record_length_field declaration first (see the negative-size error) — misframing cascades into this error.
3. Use EbcdicBackendOptions.max_records to stop at a known-good boundary while debugging, and check whether the error persists on the pristine file.
4. Confirm prefix fields (sizes and presence) match the actual feed format; remove record_length_field/record_type_field declarations if the feed has no such prefix.

Example fix

# before
# file transferred via FTP text mode -> truncated/corrupt record boundaries
conv.convert(Path('feed.dat'))  # Input ends inside 'LEN': 1 of 2 bytes left

# after
# re-transfer in binary mode, then verify expected byte count
expected = num_records * record_size
assert Path('feed.dat').stat().st_size == expected
conv.convert(Path('feed.dat'))

from pathlib import Path

def size_consistent(path: Path, record_size: int | None, declared_total: int | None) -> bool:
    n = path.stat().st_size
    if record_size is not None and n % record_size != 0:
        return False  # truncated fixed-record file
    if declared_total is not None and n != declared_total:
        return False
    return True

from docling.backend.ebcdic_backend import EbcdicDecodeError
try:
    conv.convert(src, pipeline_options=opts)
except EbcdicDecodeError as e:
    if "Input ends inside" in str(e):
        retransfer_binary(src)  # truncation: refetch, do not parse a partial file
    else:
        raise

When it happens

Trigger: A truncated EBCDIC file (partial transfer, fixed max_records cutting mid-record, network cut); a wrong record length earlier causing the next prefix read to land past the end; a layout whose prefix fields are larger than the real prefix; final record without the declared length/type prefix.

Common situations: FTP/transfers in text mode corrupting record boundaries and truncating trailing bytes; fixed-block files sliced incorrectly; upstream job writing a partial last block; layouts assuming a length prefix on records that actually have none for the final segment.

Related errors

• Cannot decode field {field.name!r} of type {field.type.value
• No record layout matches record type {record_type!r}.
• Record length {length} is shorter than the {layout.prefix_si
• Unknown EBCDIC codec {encoding!r}.
• Could not initialize the EBCDIC backend for file with hash {

AI-assisted analysis of docling-project/docling@61d76f1ff3 (2026-08-14). Data as JSON: /api/errors/5c2d77a60cb9d1b6. Report an issue: GitHub.