{"record":{"id":"68b1292603ad85dd","repo":"docling-project/docling","slug":"invalid-binary-inference-response-header-from-res","errorCode":null,"errorMessage":"Invalid binary inference response header from {response.url}: {exc}","messagePattern":"Invalid binary inference response header from (.+?): (.+?)","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"docling/models/inference_engines/common/kserve_v2_http.py","lineNumber":200,"sourceCode":"            ) from exc\n\n    try:\n        header_len = int(header_len_text)\n    except ValueError as exc:\n        raise RuntimeError(\n            f\"Invalid {_INFERENCE_HEADER_CONTENT_LENGTH} value: {header_len_text!r}\"\n        ) from exc\n\n    if header_len < 0 or header_len > len(response.content):\n        raise RuntimeError(\n            f\"Invalid {_INFERENCE_HEADER_CONTENT_LENGTH} value: {header_len}\"\n        )\n\n    try:\n        header_json = json.loads(response.content[:header_len].decode(\"utf-8\"))\n        return KserveV2InferResponse.model_validate(header_json)\n    except Exception as exc:\n        raise RuntimeError(\n            f\"Invalid binary inference response header from {response.url}: {exc}\"\n        ) from exc\n\n\n@dataclass(frozen=True)\nclass KserveV2HttpClient:\n    \"\"\"Minimal client for KServe v2 REST infer requests.\"\"\"\n\n    base_url: str\n    model_name: str\n    model_version: Optional[str]\n    timeout: float\n    headers: Mapping[str, str]\n    use_binary_data: bool = True\n\n    def close(self) -> None:\n        \"\"\"No-op close for transport parity with gRPC client.\"\"\"\n","sourceCodeStart":182,"sourceCodeEnd":218,"githubUrl":"https://github.com/docling-project/docling/blob/61d76f1ff3f8428065465889f7b4577da7df704c/docling/models/inference_engines/common/kserve_v2_http.py#L182-L218","documentation":"_decode_binary_response: the header length was valid, but decoding response.content[:header_len] as UTF-8 JSON and validating it as KserveV2InferResponse failed (json.loads, UnicodeDecodeError, or pydantic ValidationError). The chained exception carries the precise reason. The declared header region does not contain a well-formed infer response.","triggerScenarios":"Header length off by N bytes (server counting differently), so the slice cuts into/short of the JSON; binary payload preceded by non-JSON bytes; proxy injecting a prefix (BOM, banner) before the header; pydantic validation failing because outputs are missing or mistyped.","commonSituations":"Custom servers implementing the binary extension slightly wrong; encoding transformations (gzip auto-decompress changing offsets); client pydantic model stricter than what the server emits.","solutions":["Dump response.content[:header_len] in a repro script and eyeball whether it is complete JSON","Verify the server against the KServe v2 binary tensor extension spec (header length must be the exact JSON byte count)","Disable transparent gzip at proxies, or the binary mode, while diagnosing","Report to the server implementer with the captured header bytes"],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"try:\n    outputs = client.infer(inputs=inputs, output_names=[...])\nexcept RuntimeError as e:\n    if \"Invalid binary inference response header\" in str(e):\n        raise  # inspect e.__cause__ for the JSON/pydantic error; fix server header\n    raise","preventionTips":["Validate custom servers against the KServe binary extension's exact header-length semantics","Disable transparent content-encoding transformations on the infer route","Capture and diff a failing raw body when integrating a new server"],"tags":["http","kserve","binary-data","response-format","server-bug"],"backgroundTag":null,"analyzedSha":"61d76f1ff3f8428065465889f7b4577da7df704c","analyzedAt":"2026-08-14T23:53:18.727Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}