docling-project/docling · error · RuntimeError

Missing expected KServe v2 output: {self._output_name}

Error message

Missing expected KServe v2 output: {self._output_name}

What it means

The inference response dictionary did not contain the output tensor name that the engine resolved from model metadata at initialization time. Docling requested output_names=[self._output_name] and then indexes the result by that key; a KeyError means the server returned results under different names than its metadata advertised.

Source

Thrown at docling/models/inference_engines/image_classification/api_kserve_v2_engine.py:170

        # Type narrowing: _initialized guarantees these are non-None
        assert self._processor is not None
        assert self._kserve_client is not None
        assert self._input_name is not None
        assert self._output_name is not None

        images = [item.image.convert("RGB") for item in input_batch]
        processed_inputs = self._processor(images=images, return_tensors="np")
        pixel_values = np.asarray(processed_inputs["pixel_values"])

        outputs = self._kserve_client.infer(
            inputs={self._input_name: pixel_values},
            output_names=[self._output_name],
            request_parameters=self.options.request_parameters,
        )
        try:
            logits_batch = outputs[self._output_name]
        except KeyError as exc:
            raise RuntimeError(
                f"Missing expected KServe v2 output: {self._output_name}"
            ) from exc

        logits_batch = np.asarray(logits_batch, dtype=np.float32)
        if logits_batch.ndim != 2:
            raise RuntimeError(
                "Expected logits output shape [batch_size, num_classes], "
                f"got shape={logits_batch.shape}"
            )

        probs_batch = self._softmax(logits_batch)
        return self._build_batch_outputs_from_probabilities(
            input_batch=input_batch,
            probs_batch=probs_batch,
        )

    def close(self) -> None:
        if self._kserve_client is None:

View on GitHub (pinned to 61d76f1ff3)

Solutions

1. Re-create the engine (re-run initialize()) so tensor names are re-resolved against the current server state.
2. Pin options.model_version so metadata and inference hit the identical model revision.
3. Inspect the raw InferResponse output names server-side and align the server so inference responses match its advertised metadata names.
4. Avoid pointing the engine at auto-updating/canary endpoints; use a fixed version.

Example fix

# before: versionless endpoint, model swapped mid-session
options.model_version = None

# after: pin the version
options.model_version = "v1"

resp = kserve_client.infer(inputs=..., output_names=[name])
if name not in resp:
    available = list(resp.keys())
    raise ValueError(f"output '{name}' missing; server returned: {available}")

try:
    engine.predict_batch(batch)
except RuntimeError as e:
    if "Missing expected KServe v2 output" in str(e):
        engine.close()
        engine = rebuild_engine()  # re-resolve tensor names
        engine.initialize()
        engine.predict_batch(batch)
    else:
        raise

When it happens

Trigger: ApiKserveV2ImageClassificationEngine.predict_batch() when the server's InferResponse contents use an output name different from metadata.outputs[0].name (or omit it), so outputs[self._output_name] raises KeyError.

Common situations: Server restarted with a different model version between initialize() and predict(); custom server naming outputs inconsistently between metadata and inference responses; versionless endpoint being updated (canary rollout) behind the client's cached name.

Related errors

• Invalid BYTES data: insufficient bytes for string of length
• Expected image-classification model metadata to expose at le
• Expected image-classification model metadata to expose at le
• Expected logits output shape [batch_size, num_classes], got
• Preset '{preset_id}' uses API_KSERVE_V2 engine which require

AI-assisted analysis of docling-project/docling@61d76f1ff3 (2026-08-14). Data as JSON: /api/errors/c90416225102440f. Report an issue: GitHub.