docling-project/docling · error · RuntimeError

KServe v2 client is not initialized.

Error message

KServe v2 client is not initialized.

What it means

The KServe v2 engine raises RuntimeError('KServe v2 client is not initialized.') in _resolve_tensor_names when self._kserve_client is still None. The tensor names for inputs/outputs are only discovered after the KServe V2 client is created during engine initialization, so calling any inference or metadata-dependent method before initialize() is a programming error.

Source

Thrown at docling/models/inference_engines/object_detection/api_kserve_v2_engine.py:76

        if not enable_remote_services:
            raise OperationNotAllowed(
                "Connections to remote services are only allowed when set explicitly. "
                "pipeline_options.enable_remote_services=True."
            )

    def _resolve_model_name(self) -> str:
        if self.options.model_name:
            return self.options.model_name

        return self._repo_id.replace("/", "--")

    def _resolve_model_version(self) -> Optional[str]:
        return self.options.model_version

    def _resolve_tensor_names(self) -> tuple[str, str, str, str, str]:
        if self._kserve_client is None:
            raise RuntimeError("KServe v2 client is not initialized.")

        metadata = self._kserve_client.get_model_metadata()
        if len(metadata.inputs) < 2:
            raise RuntimeError(
                "Expected object-detection model metadata to expose at least 2 inputs "
                f"(images, orig_target_sizes), got {len(metadata.inputs)}."
            )
        if len(metadata.outputs) < 3:
            raise RuntimeError(
                "Expected object-detection model metadata to expose at least 3 outputs "
                f"(labels, boxes, scores), got {len(metadata.outputs)}."
            )

        input_images_name = metadata.inputs[0].name
        input_orig_target_sizes_name = metadata.inputs[1].name
        output_labels_name = metadata.outputs[0].name
        output_boxes_name = metadata.outputs[1].name
        output_scores_name = metadata.outputs[2].name

View on GitHub (pinned to 61d76f1ff3)

Solutions

1. Call engine.initialize() once after construction and before predict_batch().
2. If initialize() was already called, check logs for an earlier swallowed exception during client construction (wrong url/inference_port in ApiKserveV2ObjectDetectionEngineOptions).
3. Do not catch and discard exceptions from initialize(); treat any initialization failure as fatal for that engine instance.

Example fix

# before
engine = ApiKserveV2ObjectDetectionEngine(options=opts, ...)
outputs = engine.predict_batch(inputs)

# after
engine = ApiKserveV2ObjectDetectionEngine(options=opts, ...)
engine.initialize()
outputs = engine.predict_batch(inputs)

if engine._kserve_client is None:
    engine.initialize()

try:
    engine.initialize()
except Exception:
    log.exception("KServe init failed; check url/port and network")
    raise

When it happens

Trigger: Instantiating ApiKserveV2ObjectDetectionEngine and calling predict_batch() (or anything that reaches _resolve_tensor_names) without calling engine.initialize() first; or initialize() having failed partway leaving the client unset while the exception was swallowed.

Common situations: Custom orchestration code that manages engine lifecycles manually and skips initialize(); a previous initialize() failure (bad URL, TLS error) being caught and ignored, then predict being attempted anyway.

Related errors

• Engine not initialized. Call initialize() first.
• KServe v2 client is not initialized.
• Engine not initialized. Call initialize() first.
• Engine not initialized. Call initialize() first.
• Engine not initialized. Call initialize() first.

AI-assisted analysis of docling-project/docling@61d76f1ff3 (2026-08-14). Data as JSON: /api/errors/ba05b4c1d1b62968. Report an issue: GitHub.