{"record":{"id":"11ef5ca0c45b64c3","repo":"docling-project/docling","slug":"missing-one-or-more-expected-kserve-v2-outputs-s","errorCode":null,"errorMessage":"Missing one or more expected KServe v2 outputs: {self._output_labels_name}, {self._output_boxes_name}, {self._output_scores_name}","messagePattern":"Missing one or more expected KServe v2 outputs: (.+?), (.+?), (.+?)","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"docling/models/inference_engines/object_detection/api_kserve_v2_engine.py","lineNumber":221,"sourceCode":"\n        outputs = self._kserve_client.infer(\n            inputs={\n                self._input_images_name: pixel_values,\n                self._input_orig_target_sizes_name: orig_sizes,\n            },\n            output_names=[\n                self._output_labels_name,\n                self._output_boxes_name,\n                self._output_scores_name,\n            ],\n            request_parameters=self.options.request_parameters,\n        )\n        try:\n            labels_batch = outputs[self._output_labels_name]\n            boxes_batch = outputs[self._output_boxes_name]\n            scores_batch = outputs[self._output_scores_name]\n        except KeyError as exc:\n            raise RuntimeError(\n                \"Missing one or more expected KServe v2 outputs: \"\n                f\"{self._output_labels_name}, \"\n                f\"{self._output_boxes_name}, \"\n                f\"{self._output_scores_name}\"\n            ) from exc\n\n        if len(labels_batch) != len(input_batch):\n            raise RuntimeError(\n                \"KServe v2 output batch size mismatch for labels: \"\n                f\"expected {len(input_batch)}, got {len(labels_batch)}\"\n            )\n\n        batch_outputs: List[ObjectDetectionEngineOutput] = []\n        for idx, input_item in enumerate(input_batch):\n            batch_outputs.append(\n                self._build_output(\n                    input_item=input_item,\n                    labels=labels_batch[idx],","sourceCodeStart":203,"sourceCodeEnd":239,"githubUrl":"https://github.com/docling-project/docling/blob/61d76f1ff3f8428065465889f7b4577da7df704c/docling/models/inference_engines/object_detection/api_kserve_v2_engine.py#L203-L239","documentation":"After a KServe v2 inference call, the engine indexes the response dict by the output tensor names discovered from model metadata (labels, boxes, scores). A KeyError means the server's response payload does not contain one or more of those names, so the engine wraps it in RuntimeError with the missing names listed.","triggerScenarios":"The KServe server's infer response omits an expected output tensor — typically because the served model version changed between metadata discovery and inference, or the server wraps outputs under different names (e.g. prefixed with the model name).","commonSituations":"The endpoint's model was swapped/redeployed mid-session; a Triton model repository where outputs are aliased; a proxy (Seldon, KServe transformer) that reshapes the response; stale tensor names cached from initialize() across a server restart.","solutions":["Re-instantiate / re-initialize the engine so tensor names are re-resolved against the current server state.","Inspect the raw KServe v2 infer response to see the actual output names and compare with the metadata; fix the server-side model config so names match.","Pin a specific model_version in ApiKserveV2ObjectDetectionEngineOptions so metadata and inference hit the same model."],"exampleFix":"# before\nopts = ApiKserveV2ObjectDetectionEngineOptions(url=url)  # version drifts with server updates\n\n# after\nopts = ApiKserveV2ObjectDetectionEngineOptions(url=url, model_version=\"v1\")  # pinned, metadata and infer agree","handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"try:\n    outputs = engine.predict_batch(batch)\nexcept RuntimeError as e:\n    if \"Missing one or more expected KServe v2 outputs\" in str(e):\n        engine = rebuild_engine()  # re-resolve tensor names against current server\n        outputs = engine.predict_batch(batch)\n    else:\n        raise","preventionTips":["Pin model_version so metadata and inference cannot diverge across deploys.","Coordinate server model updates with engine restarts.","Monitor server response shapes in integration tests."],"tags":["kserve","model-mismatch","inference","remote-services"],"backgroundTag":null,"analyzedSha":"61d76f1ff3f8428065465889f7b4577da7df704c","analyzedAt":"2026-08-14T23:53:18.727Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}