docling-project/docling · error · EbcdicDecodeError

Record length {length} is shorter than the {layout.prefix_si

Error message

Record length {length} is shorter than the {layout.prefix_size}-byte record prefix.

What it means

EbcdicDecodeError raised when the decoded record length is smaller than the layout's own prefix: size = length - prefix_size came out negative, meaning the length field decoded to a value below the number of bytes already consumed by the length/type prefix. This almost always means the length field itself is being decoded incorrectly (wrong signedness, size, scale, or byte position), because a real mainframe record length is never below its prefix size.

Source

Thrown at docling/backend/ebcdic_backend.py:173

        if (field := layout.record_length_field) is not None:
            chunk = self._take(data, offset, field.size, end, field.name)
            length = int(self._decoder.decode(chunk, field))
            offset += field.size
        if (field := layout.record_type_field) is not None:
            chunk = self._take(data, offset, field.size, end, field.name)
            record_type = str(self._decoder.decode(chunk, field))
            offset += field.size

        record = layout.select(record_type)
        if record is None:
            raise EbcdicDecodeError(
                f"No record layout matches record type {record_type!r}."
            )

        size = record.size if length is None else length - layout.prefix_size
        if size < 0:
            raise EbcdicDecodeError(
                f"Record length {length} is shorter than the "
                f"{layout.prefix_size}-byte record prefix."
            )
        return record, size, offset

    @staticmethod
    def _take(data: bytes, offset: int, size: int, end: int, name: str) -> bytes:
        if offset + size > end:
            raise EbcdicDecodeError(
                f"Input ends inside {name!r}: {end - offset} of {size} bytes left."
            )
        return data[offset : offset + size]

    def _decode_record(self, record: EbcdicRecordLayout, body: bytes) -> list[str]:
        values: list[str] = []
        offset = 0
        for field in record.fields:
            chunk = body[offset : offset + field.size]

View on GitHub (pinned to 61d76f1ff3)

Solutions

Check the decoded length value by decoding the same field manually from the hex bytes and reconcile it with the actual record size in the file.
Fix record_length_field declaration: correct type (UNSIGNED_INTEGER vs INTEGER vs ZONED/PACKED), size, and offset.
If the length unit is halfwords or excludes the prefix, transform it (scale or offset the decoded value) rather than feeding it raw — verify against prefix_size.
Ensure layout.prefix_size matches the real prefix byte count of your feed.

Example fix

# before
record_length_field=EbcdicField(name='LEN', type=EbcdicFieldType.INTEGER, size=2)
# value 0x8001-style high-bit length decodes negative -> size < 0

# after
record_length_field=EbcdicField(name='LEN', type=EbcdicFieldType.UNSIGNED_INTEGER, size=2)

Defensive patterns

Strategy: validation

Validate before calling

# Validate length decoding against reality before batch:
def length_field_sane(layout, data: bytes) -> bool:
    for offset, actual_size in iter_real_records(data):  # ground truth framing
        raw = data[offset + layout.record_length_field.offset:
                   offset + layout.record_length_field.offset + layout.record_length_field.size]
        decoded = int(decode_field(raw, layout.record_length_field))
        if decoded < layout.prefix_size or decoded != actual_size:
            return False
    return True

Try / catch

from docling.backend.ebcdic_backend import EbcdicDecodeError
try:
    conv.convert(src, pipeline_options=opts)
except EbcdicDecodeError as e:
    if "shorter than the" in str(e) and "record prefix" in str(e):
        opts.layout.record_length_field.type = EbcdicFieldType.UNSIGNED_INTEGER  # common fix
        conv.convert(src, pipeline_options=opts)
    else:
        raise

Prevention

Check signedness of length fields (mainframe lengths are unsigned).
Confirm whether lengths include the prefix and their unit (bytes vs halfwords).
Cross-check a few decoded lengths against actual record sizes on a sample file.

When it happens

Trigger: record_length_field declared as signed INTEGER when the value is unsigned (or vice versa) producing a negative decode; length field size/offset wrong so unrelated bytes are read as the length; length encoded as packed/zoned decimal but declared as binary INTEGER; unit confusion — length counted in halfwords (x2) or excluding the length field itself, so a valid length decodes to a tiny number.

Common situations: Copybook BDW/SDW lengths in halfwords translated literally; RECORD LENGTH semantics differing between IBM utilities (includes vs excludes the RDW/prefix); layouts ported from a different feeder system with a different prefix convention.

Related errors

AI-assisted analysis of docling-project/docling@61d76f1ff3 (2026-08-14). Data as JSON: /api/errors/d0b7c2365d2906a9. Report an issue: GitHub.