docling-project/docling · error · RuntimeError

Missing one or more expected KServe v2 outputs: {self._outpu

Error message

Missing one or more expected KServe v2 outputs: {self._output_labels_name}, {self._output_boxes_name}, {self._output_scores_name}

What it means

After a KServe v2 inference call, the engine indexes the response dict by the output tensor names discovered from model metadata (labels, boxes, scores). A KeyError means the server's response payload does not contain one or more of those names, so the engine wraps it in RuntimeError with the missing names listed.

Source

Thrown at docling/models/inference_engines/object_detection/api_kserve_v2_engine.py:221

        outputs = self._kserve_client.infer(
            inputs={
                self._input_images_name: pixel_values,
                self._input_orig_target_sizes_name: orig_sizes,
            },
            output_names=[
                self._output_labels_name,
                self._output_boxes_name,
                self._output_scores_name,
            ],
            request_parameters=self.options.request_parameters,
        )
        try:
            labels_batch = outputs[self._output_labels_name]
            boxes_batch = outputs[self._output_boxes_name]
            scores_batch = outputs[self._output_scores_name]
        except KeyError as exc:
            raise RuntimeError(
                "Missing one or more expected KServe v2 outputs: "
                f"{self._output_labels_name}, "
                f"{self._output_boxes_name}, "
                f"{self._output_scores_name}"
            ) from exc

        if len(labels_batch) != len(input_batch):
            raise RuntimeError(
                "KServe v2 output batch size mismatch for labels: "
                f"expected {len(input_batch)}, got {len(labels_batch)}"
            )

        batch_outputs: List[ObjectDetectionEngineOutput] = []
        for idx, input_item in enumerate(input_batch):
            batch_outputs.append(
                self._build_output(
                    input_item=input_item,
                    labels=labels_batch[idx],

View on GitHub (pinned to 61d76f1ff3)

Solutions

1. Re-instantiate / re-initialize the engine so tensor names are re-resolved against the current server state.
2. Inspect the raw KServe v2 infer response to see the actual output names and compare with the metadata; fix the server-side model config so names match.
3. Pin a specific model_version in ApiKserveV2ObjectDetectionEngineOptions so metadata and inference hit the same model.

Example fix

# before
opts = ApiKserveV2ObjectDetectionEngineOptions(url=url)  # version drifts with server updates

# after
opts = ApiKserveV2ObjectDetectionEngineOptions(url=url, model_version="v1")  # pinned, metadata and infer agree

try:
    outputs = engine.predict_batch(batch)
except RuntimeError as e:
    if "Missing one or more expected KServe v2 outputs" in str(e):
        engine = rebuild_engine()  # re-resolve tensor names against current server
        outputs = engine.predict_batch(batch)
    else:
        raise

When it happens

Trigger: The KServe server's infer response omits an expected output tensor — typically because the served model version changed between metadata discovery and inference, or the server wraps outputs under different names (e.g. prefixed with the model name).

Common situations: The endpoint's model was swapped/redeployed mid-session; a Triton model repository where outputs are aliased; a proxy (Seldon, KServe transformer) that reshapes the response; stale tensor names cached from initialize() across a server restart.

Related errors

• Expected object-detection model metadata to expose at least
• Invalid inference response from {self.infer_url}: {exc}
• Connections to remote services are only allowed when set exp
• Connections to remote services are only allowed when set exp
• Expected object-detection model metadata to expose at least

AI-assisted analysis of docling-project/docling@61d76f1ff3 (2026-08-14). Data as JSON: /api/errors/11ef5ca0c45b64c3. Report an issue: GitHub.