{"record":{"id":"a01faf1ea7e72394","repo":"docling-project/docling","slug":"binary-kserve-response-from-response-url-did-not","errorCode":null,"errorMessage":"Binary KServe response from {response.url} did not include {_INFERENCE_HEADER_CONTENT_LENGTH} and was not valid JSON: {exc}","messagePattern":"Binary KServe response from (.+?) did not include (.+?) and was not valid JSON: (.+?)","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"docling/models/inference_engines/common/kserve_v2_http.py","lineNumber":179,"sourceCode":"    if request_parameters:\n        payload[\"parameters\"] = dict(request_parameters)\n\n    header_bytes = json.dumps(payload, separators=(\",\", \":\")).encode(\"utf-8\")\n    request_body = header_bytes + b\"\".join(raw_inputs)\n    request_headers = {\n        \"Content-Type\": \"application/octet-stream\",\n        _INFERENCE_HEADER_CONTENT_LENGTH: str(len(header_bytes)),\n    }\n    return request_headers, request_body\n\n\ndef _decode_binary_response(response: requests.Response) -> KserveV2InferResponse:\n    header_len_text = response.headers.get(_INFERENCE_HEADER_CONTENT_LENGTH)\n    if header_len_text is None:\n        try:\n            return KserveV2InferResponse.model_validate(response.json())\n        except Exception as exc:\n            raise RuntimeError(\n                f\"Binary KServe response from {response.url} did not include \"\n                f\"{_INFERENCE_HEADER_CONTENT_LENGTH} and was not valid JSON: {exc}\"\n            ) from exc\n\n    try:\n        header_len = int(header_len_text)\n    except ValueError as exc:\n        raise RuntimeError(\n            f\"Invalid {_INFERENCE_HEADER_CONTENT_LENGTH} value: {header_len_text!r}\"\n        ) from exc\n\n    if header_len < 0 or header_len > len(response.content):\n        raise RuntimeError(\n            f\"Invalid {_INFERENCE_HEADER_CONTENT_LENGTH} value: {header_len}\"\n        )\n\n    try:\n        header_json = json.loads(response.content[:header_len].decode(\"utf-8\"))","sourceCodeStart":161,"sourceCodeEnd":197,"githubUrl":"https://github.com/docling-project/docling/blob/61d76f1ff3f8428065465889f7b4577da7df704c/docling/models/inference_engines/common/kserve_v2_http.py#L161-L197","documentation":"In _decode_binary_response: the response lacks the Inference-Header-Content-Length header (which should delimit the JSON header preceding the binary payload), so the client falls back to parsing the whole body as a plain JSON infer response - and that also failed. The chained exception says why the JSON parse/validation failed. Typically the body is an error page/JSON, or a proxy stripped the header.","triggerScenarios":"use_binary_data=True but the server returned a JSON error (4xx/5xx body that was still 200 via gateway, or a plain error payload); an intermediary (nginx, API gateway) dropped unknown headers; the endpoint is not actually a KServe v2 infer route.","commonSituations":"Base URL pointing at the wrong path (e.g. root instead of /v2/models/<m>/infer); auth failures surfaced as JSON error bodies; header-filtering proxies in front of the model server.","solutions":["Replay the request with curl -i and inspect status, headers, and body - almost always the body is an error message, and fixing that error resolves this","Ensure the gateway forwards the Inference-Header-Content-Length response header (no underspecified header whitelisting)","Confirm base_url/model_name build the correct /v2/models/<model>/infer path","If the server genuinely does not support binary responses, set use_binary_data=False"],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"try:\n    outputs = client.infer(inputs=inputs, output_names=[...])\nexcept RuntimeError as e:\n    if \"did not include Inference-Header-Content-Length\" in str(e):\n        # body is usually a server error - inspect it via a debug proxy and fix root cause\n        raise\n    raise","preventionTips":["Verify base_url points to a real KServe v2 infer route with curl before wiring the client","Configure gateways to forward/allow the Inference-Header-Content-Length response header","Surface server-side errors distinctly so a 200-with-error-body is caught early"],"tags":["http","kserve","binary-data","gateway","response-format"],"backgroundTag":null,"analyzedSha":"61d76f1ff3f8428065465889f7b4577da7df704c","analyzedAt":"2026-08-14T23:53:18.727Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}