docling-project/docling · error · RuntimeError

Missing one or more expected KServe v2 outputs

Error message

Missing one or more expected KServe v2 outputs: {self._output_labels_name}, {self._output_boxes_name}, {self._output_scores_name}

What it means

After a KServe v2 inference call, the engine indexes the response dict by the output tensor names discovered from model metadata (labels, boxes, scores). A KeyError means the server's response payload does not contain one or more of those names, so the engine wraps it in RuntimeError with the missing names listed.

Solutions

Re-instantiate / re-initialize the engine so tensor names are re-resolved against the current server state.
Inspect the raw KServe v2 infer response to see the actual output names and compare with the metadata; fix the server-side model config so names match.
Pin a specific model_version in ApiKserveV2ObjectDetectionEngineOptions so metadata and inference hit the same model.

Example fix

# before
opts = ApiKserveV2ObjectDetectionEngineOptions(url=url)  # version drifts with server updates

# after
opts = ApiKserveV2ObjectDetectionEngineOptions(url=url, model_version="v1")  # pinned, metadata and infer agree

Defensive patterns

Strategy: retry

Try / catch

try:
    outputs = engine.predict_batch(batch)
except RuntimeError as e:
    if "Missing one or more expected KServe v2 outputs" in str(e):
        engine = rebuild_engine()  # re-resolve tensor names against current server
        outputs = engine.predict_batch(batch)
    else:
        raise

Prevention

Pin model_version so metadata and inference cannot diverge across deploys.
Coordinate server model updates with engine restarts.
Monitor server response shapes in integration tests.

When it happens

Trigger: The KServe server's infer response omits an expected output tensor — typically because the served model version changed between metadata discovery and inference, or the server wraps outputs under different names (e.g. prefixed with the model name).

Common situations: The endpoint's model was swapped/redeployed mid-session; a Triton model repository where outputs are aliased; a proxy (Seldon, KServe transformer) that reshapes the response; stale tensor names cached from initialize() across a server restart.

Related errors

AI-assisted analysis of docling-project/docling@61d76f1ff3 (2026-08-14). Data as JSON: /api/errors/11ef5ca0c45b64c3. Report an issue: GitHub.

Appendix: source

Thrown at docling/models/inference_engines/object_detection/api_kserve_v2_engine.py:221

        outputs = self._kserve_client.infer(
            inputs={
                self._input_images_name: pixel_values,
                self._input_orig_target_sizes_name: orig_sizes,
            },
            output_names=[
                self._output_labels_name,
                self._output_boxes_name,
                self._output_scores_name,
            ],
            request_parameters=self.options.request_parameters,
        )
        try:
            labels_batch = outputs[self._output_labels_name]
            boxes_batch = outputs[self._output_boxes_name]
            scores_batch = outputs[self._output_scores_name]
        except KeyError as exc:
            raise RuntimeError(
                "Missing one or more expected KServe v2 outputs: "
                f"{self._output_labels_name}, "
                f"{self._output_boxes_name}, "
                f"{self._output_scores_name}"
            ) from exc

        if len(labels_batch) != len(input_batch):
            raise RuntimeError(
                "KServe v2 output batch size mismatch for labels: "
                f"expected {len(input_batch)}, got {len(labels_batch)}"
            )

        batch_outputs: List[ObjectDetectionEngineOutput] = []
        for idx, input_item in enumerate(input_batch):
            batch_outputs.append(
                self._build_output(
                    input_item=input_item,
                    labels=labels_batch[idx],

View on GitHub (pinned to 61d76f1ff3)