{"record":{"id":"1eb5a0830fe8a303","repo":"docling-project/docling","slug":"kserve-v2-http-response-included-trailing-binary-o","errorCode":null,"errorMessage":"KServe v2 HTTP response included trailing binary output data that was not consumed: {len(raw_body) - raw_offset} bytes","messagePattern":"KServe v2 HTTP response included trailing binary output data that was not consumed: (.+?) bytes","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"docling/models/inference_engines/common/kserve_v2_http.py","lineNumber":431,"sourceCode":"            binary_data_size = _parse_binary_data_size(output.parameters)\n            if binary_data_size is None:\n                decoded_outputs[output.name] = _decode_output_tensor(output)\n                continue\n\n            raw_end = raw_offset + binary_data_size\n            if raw_end > len(raw_body):\n                raise RuntimeError(\n                    \"KServe v2 HTTP response did not include enough binary output data \"\n                    f\"for tensor {output.name}: expected {binary_data_size} bytes at \"\n                    f\"offset {raw_offset}, got {len(raw_body) - raw_offset}\"\n                )\n            decoded_outputs[output.name] = _decode_binary_output_tensor(\n                output, raw_body[raw_offset:raw_end]\n            )\n            raw_offset = raw_end\n\n        if raw_offset != len(raw_body):\n            raise RuntimeError(\n                \"KServe v2 HTTP response included trailing binary output data that was \"\n                f\"not consumed: {len(raw_body) - raw_offset} bytes\"\n            )\n\n        if _log.isEnabledFor(logging.DEBUG):\n            _log.debug(\n                \"PIPELINE_PROFILING KServe infer deserialization: batch_size=%d start=%.3f end=%.3f duration=%.3fs\",\n                _batch_size,\n                _t_deser_start,\n                time.time(),\n                time.monotonic() - _t_deser_mono,\n            )\n\n        return decoded_outputs\n","sourceCodeStart":413,"sourceCodeEnd":446,"githubUrl":"https://github.com/docling-project/docling/blob/61d76f1ff3f8428065465889f7b4577da7df704c/docling/models/inference_engines/common/kserve_v2_http.py#L413-L446","documentation":"After consuming all tensors that declared binary_data_size, bytes remain unconsumed in the binary body (raw_offset != len(raw_body)). The KServe v2 binary extension requires the binary payloads to be exactly the concatenation of the declared per-tensor sizes; leftover bytes mean the sizes undercount what was sent, or the header-length slicing misaligned the body start.","triggerScenarios":"Server understates one or more binary_data_size values; an extra binary payload appended for a tensor that did not declare one; Inference-Header-Content-Length overcounting so raw_body starts too early; predictor emitting padding bytes.","commonSituations":"Custom KServe runtimes with sloppy size accounting; padding/alignment added by an intermediary; evolving server code during a rolling deploy.","solutions":["Compare the sum of declared binary_data_size values with len(raw_body) to confirm the accounting gap","Fix the server-side binary_data_size computation to match the bytes actually written","Check the Inference-Header-Content-Length value against the true JSON header length","Disable binary transport while the server's binary implementation is non-conformant"],"exampleFix":null,"handlingStrategy":"fallback","validationCode":null,"typeGuard":null,"tryCatchPattern":"try:\n    outputs = client.infer(inputs=inputs, output_names=[...])\nexcept RuntimeError as e:\n    if \"trailing binary output data\" in str(e):\n        client = replace(client, use_binary_data=False)\n        outputs = client.infer(inputs=inputs, output_names=[...])\n    else:\n        raise","preventionTips":["Run the server's own binary-tensor conformance tests after predictor changes","Cross-check declared binary_data_size sums against actual payload sizes in integration tests","Avoid binary mode against servers with hand-rolled serialization"],"tags":["http","kserve","binary-data","server-bug","protocol-mismatch"],"backgroundTag":null,"analyzedSha":"61d76f1ff3f8428065465889f7b4577da7df704c","analyzedAt":"2026-08-14T23:53:18.727Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}