{"record":{"id":"f4c234031241872a","repo":"docling-project/docling","slug":"kserve-v2-http-response-did-not-include-enough-bin","errorCode":null,"errorMessage":"KServe v2 HTTP response did not include enough binary output data for tensor {output.name}: expected {binary_data_size} bytes at offset {raw_offset}, got {len(raw_body) - raw_offset}","messagePattern":"KServe v2 HTTP response did not include enough binary output data for tensor (.+?): expected (.+?) bytes at offset (.+?), got (.+?)","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"docling/models/inference_engines/common/kserve_v2_http.py","lineNumber":420,"sourceCode":"            raise RuntimeError(\n                f\"Invalid inference response from {self.infer_url}: {exc}\"\n            ) from exc\n\n        decoded_outputs: Dict[str, np.ndarray] = {}\n        header_len_text = response.headers.get(_INFERENCE_HEADER_CONTENT_LENGTH)\n        raw_body = b\"\"\n        if self.use_binary_data and header_len_text is not None:\n            raw_body = response.content[int(header_len_text) :]\n        raw_offset = 0\n        for output in body.outputs:\n            binary_data_size = _parse_binary_data_size(output.parameters)\n            if binary_data_size is None:\n                decoded_outputs[output.name] = _decode_output_tensor(output)\n                continue\n\n            raw_end = raw_offset + binary_data_size\n            if raw_end > len(raw_body):\n                raise RuntimeError(\n                    \"KServe v2 HTTP response did not include enough binary output data \"\n                    f\"for tensor {output.name}: expected {binary_data_size} bytes at \"\n                    f\"offset {raw_offset}, got {len(raw_body) - raw_offset}\"\n                )\n            decoded_outputs[output.name] = _decode_binary_output_tensor(\n                output, raw_body[raw_offset:raw_end]\n            )\n            raw_offset = raw_end\n\n        if raw_offset != len(raw_body):\n            raise RuntimeError(\n                \"KServe v2 HTTP response included trailing binary output data that was \"\n                f\"not consumed: {len(raw_body) - raw_offset} bytes\"\n            )\n\n        if _log.isEnabledFor(logging.DEBUG):\n            _log.debug(\n                \"PIPELINE_PROFILING KServe infer deserialization: batch_size=%d start=%.3f end=%.3f duration=%.3fs\",","sourceCodeStart":402,"sourceCodeEnd":438,"githubUrl":"https://github.com/docling-project/docling/blob/61d76f1ff3f8428065465889f7b4577da7df704c/docling/models/inference_engines/common/kserve_v2_http.py#L402-L438","documentation":"During binary HTTP response slicing: the header JSON declared a tensor with binary_data_size N at cumulative offset raw_offset, but the remaining body after the JSON header is shorter than N bytes. The client refuses to read past the buffer. Indicates truncated payloads or binary_data_size values inconsistent with what was actually sent.","triggerScenarios":"Response truncated in transit (proxy size limit, connection cut); server computing binary_data_size before some transformation (gzip re-encode) changes byte counts; multiple binary tensors where an earlier size is overstated, shifting the budget; Inference-Header-Content-Length undercounting the JSON header so raw_body starts mid-JSON.","commonSituations":"Large vision/LLM outputs hitting gateway body limits; custom predictors with off-by-N size math; content-encoding transformations applied by CDNs.","solutions":["Log len(response.content), the header length, and each tensor's binary_data_size to find which number is wrong","Remove response-size limits and transparent content-encoding on the proxy path","If server-side sizes are wrong, fix the predictor; if transport truncates, fix the network layer","Temporarily set use_binary_data=False to keep working while diagnosing"],"exampleFix":null,"handlingStrategy":"fallback","validationCode":null,"typeGuard":null,"tryCatchPattern":"try:\n    outputs = client.infer(inputs=inputs, output_names=[...])\nexcept RuntimeError as e:\n    if \"did not include enough binary output data\" in str(e):\n        client = replace(client, use_binary_data=False)\n        outputs = client.infer(inputs=inputs, output_names=[...])\n    else:\n        raise","preventionTips":["Keep proxy response-size limits comfortably above peak binary payload sizes","Verify server binary_data_size math with a conformance test client","Shrink batches if payloads approach transport limits"],"tags":["http","kserve","binary-data","truncation","server-bug"],"backgroundTag":null,"analyzedSha":"61d76f1ff3f8428065465889f7b4577da7df704c","analyzedAt":"2026-08-14T23:53:18.727Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}