{"record":{"id":"f90833c72aacbe43","repo":"docling-project/docling","slug":"invalid-inference-response-from-self-infer-url","errorCode":null,"errorMessage":"Invalid inference response from {self.infer_url}: {exc}","messagePattern":"Invalid inference response from (.+?): (.+?)","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"docling/models/inference_engines/common/kserve_v2_http.py","lineNumber":402,"sourceCode":"        if _log.isEnabledFor(logging.DEBUG):\n            _log.debug(\n                \"PIPELINE_PROFILING KServe infer http round-trip: batch_size=%d start=%.3f end=%.3f duration=%.3fs\",\n                _batch_size,\n                _t_http_start,\n                time.time(),\n                time.monotonic() - _t_http_mono,\n            )\n            _t_deser_start = time.time()\n            _t_deser_mono = time.monotonic()\n\n        try:\n            body = (\n                _decode_binary_response(response)\n                if self.use_binary_data\n                else KserveV2InferResponse.model_validate(response.json())\n            )\n        except Exception as exc:\n            raise RuntimeError(\n                f\"Invalid inference response from {self.infer_url}: {exc}\"\n            ) from exc\n\n        decoded_outputs: Dict[str, np.ndarray] = {}\n        header_len_text = response.headers.get(_INFERENCE_HEADER_CONTENT_LENGTH)\n        raw_body = b\"\"\n        if self.use_binary_data and header_len_text is not None:\n            raw_body = response.content[int(header_len_text) :]\n        raw_offset = 0\n        for output in body.outputs:\n            binary_data_size = _parse_binary_data_size(output.parameters)\n            if binary_data_size is None:\n                decoded_outputs[output.name] = _decode_output_tensor(output)\n                continue\n\n            raw_end = raw_offset + binary_data_size\n            if raw_end > len(raw_body):\n                raise RuntimeError(","sourceCodeStart":384,"sourceCodeEnd":420,"githubUrl":"https://github.com/docling-project/docling/blob/61d76f1ff3f8428065465889f7b4577da7df704c/docling/models/inference_engines/common/kserve_v2_http.py#L384-L420","documentation":"Top-level guard in the HTTP infer path: whatever the response was (binary decode via _decode_binary_response or plain JSON validation) could not be turned into a KserveV2InferResponse, and the client wraps that failure with the infer URL. The original error is chained; this message tells you the problem is the response payload, not the request encoding.","triggerScenarios":"Server returned a JSON error object on 200; body is HTML from a gateway; binary response with a broken Inference-Header-Content-Length (see the chained _decode_binary_response errors); pydantic rejecting the outputs structure; empty body.","commonSituations":"Gateways that map upstream 5xx to 200 with an error JSON; wrong infer URL path; model crash mid-request returning a stack-trace body; version-skewed response schema.","solutions":["Inspect exc.__cause__ - it carries the specific decode/validation failure; fix that first","Reproduce the POST with curl and examine status and body","Verify the infer URL and that the model is actually loaded and healthy on the server","If a gateway rewrites status codes or bodies, exclude the infer route from it"],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"try:\n    outputs = client.infer(inputs=inputs, output_names=[...])\nexcept RuntimeError as e:\n    if \"Invalid inference response\" in str(e):\n        cause = e.__cause__  # real decode/validation failure\n        raise RuntimeError(f\"infer response unusable: {cause}\") from cause\n    raise","preventionTips":["Always inspect the chained cause; it pinpoints the actual response problem","Smoke-test one inference after any server or gateway change","Do not route model traffic through components that rewrite bodies or status codes"],"tags":["http","kserve","response-format","gateway","inference"],"backgroundTag":null,"analyzedSha":"61d76f1ff3f8428065465889f7b4577da7df704c","analyzedAt":"2026-08-14T23:53:18.727Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}