{"record":{"id":"d0b7c2365d2906a9","repo":"docling-project/docling","slug":"record-length-length-is-shorter-than-the-layout","errorCode":null,"errorMessage":"Record length {length} is shorter than the {layout.prefix_size}-byte record prefix.","messagePattern":"Record length (.+?) is shorter than the (.+?)-byte record prefix\\.","errorType":"exception","errorClass":"EbcdicDecodeError","httpStatus":null,"severity":"error","filePath":"docling/backend/ebcdic_backend.py","lineNumber":173,"sourceCode":"\n        if (field := layout.record_length_field) is not None:\n            chunk = self._take(data, offset, field.size, end, field.name)\n            length = int(self._decoder.decode(chunk, field))\n            offset += field.size\n        if (field := layout.record_type_field) is not None:\n            chunk = self._take(data, offset, field.size, end, field.name)\n            record_type = str(self._decoder.decode(chunk, field))\n            offset += field.size\n\n        record = layout.select(record_type)\n        if record is None:\n            raise EbcdicDecodeError(\n                f\"No record layout matches record type {record_type!r}.\"\n            )\n\n        size = record.size if length is None else length - layout.prefix_size\n        if size < 0:\n            raise EbcdicDecodeError(\n                f\"Record length {length} is shorter than the \"\n                f\"{layout.prefix_size}-byte record prefix.\"\n            )\n        return record, size, offset\n\n    @staticmethod\n    def _take(data: bytes, offset: int, size: int, end: int, name: str) -> bytes:\n        if offset + size > end:\n            raise EbcdicDecodeError(\n                f\"Input ends inside {name!r}: {end - offset} of {size} bytes left.\"\n            )\n        return data[offset : offset + size]\n\n    def _decode_record(self, record: EbcdicRecordLayout, body: bytes) -> list[str]:\n        values: list[str] = []\n        offset = 0\n        for field in record.fields:\n            chunk = body[offset : offset + field.size]","sourceCodeStart":155,"sourceCodeEnd":191,"githubUrl":"https://github.com/docling-project/docling/blob/61d76f1ff3f8428065465889f7b4577da7df704c/docling/backend/ebcdic_backend.py#L155-L191","documentation":"EbcdicDecodeError raised when the decoded record length is smaller than the layout's own prefix: size = length - prefix_size came out negative, meaning the length field decoded to a value below the number of bytes already consumed by the length/type prefix. This almost always means the length field itself is being decoded incorrectly (wrong signedness, size, scale, or byte position), because a real mainframe record length is never below its prefix size.","triggerScenarios":"record_length_field declared as signed INTEGER when the value is unsigned (or vice versa) producing a negative decode; length field size/offset wrong so unrelated bytes are read as the length; length encoded as packed/zoned decimal but declared as binary INTEGER; unit confusion — length counted in halfwords (x2) or excluding the length field itself, so a valid length decodes to a tiny number.","commonSituations":"Copybook BDW/SDW lengths in halfwords translated literally; RECORD LENGTH semantics differing between IBM utilities (includes vs excludes the RDW/prefix); layouts ported from a different feeder system with a different prefix convention.","solutions":["Check the decoded length value by decoding the same field manually from the hex bytes and reconcile it with the actual record size in the file.","Fix record_length_field declaration: correct type (UNSIGNED_INTEGER vs INTEGER vs ZONED/PACKED), size, and offset.","If the length unit is halfwords or excludes the prefix, transform it (scale or offset the decoded value) rather than feeding it raw — verify against prefix_size.","Ensure layout.prefix_size matches the real prefix byte count of your feed."],"exampleFix":"# before\nrecord_length_field=EbcdicField(name='LEN', type=EbcdicFieldType.INTEGER, size=2)\n# value 0x8001-style high-bit length decodes negative -> size < 0\n\n# after\nrecord_length_field=EbcdicField(name='LEN', type=EbcdicFieldType.UNSIGNED_INTEGER, size=2)","handlingStrategy":"validation","validationCode":"# Validate length decoding against reality before batch:\ndef length_field_sane(layout, data: bytes) -> bool:\n    for offset, actual_size in iter_real_records(data):  # ground truth framing\n        raw = data[offset + layout.record_length_field.offset:\n                   offset + layout.record_length_field.offset + layout.record_length_field.size]\n        decoded = int(decode_field(raw, layout.record_length_field))\n        if decoded < layout.prefix_size or decoded != actual_size:\n            return False\n    return True","typeGuard":null,"tryCatchPattern":"from docling.backend.ebcdic_backend import EbcdicDecodeError\ntry:\n    conv.convert(src, pipeline_options=opts)\nexcept EbcdicDecodeError as e:\n    if \"shorter than the\" in str(e) and \"record prefix\" in str(e):\n        opts.layout.record_length_field.type = EbcdicFieldType.UNSIGNED_INTEGER  # common fix\n        conv.convert(src, pipeline_options=opts)\n    else:\n        raise","preventionTips":["Check signedness of length fields (mainframe lengths are unsigned).","Confirm whether lengths include the prefix and their unit (bytes vs halfwords).","Cross-check a few decoded lengths against actual record sizes on a sample file."],"tags":["ebcdic","layout","record-length","decode"],"backgroundTag":null,"analyzedSha":"61d76f1ff3f8428065465889f7b4577da7df704c","analyzedAt":"2026-08-14T23:53:18.727Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}