{"record":{"id":"05ad4550f6cb45ec","repo":"docling-project/docling","slug":"grpc-infer-call-failed-for-model-self-model-name","errorCode":null,"errorMessage":"gRPC infer call failed for model {self.model_name}: {exc}","messagePattern":"gRPC infer call failed for model (.+?): (.+?)","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"docling/models/inference_engines/common/kserve_v2_grpc.py","lineNumber":347,"sourceCode":"        if _log.isEnabledFor(logging.DEBUG):\n            _log.debug(\n                \"PIPELINE_PROFILING KServe gRPC infer serialization: batch_size=%d start=%.3f end=%.3f duration=%.3fs\",\n                _batch_size,\n                _t_ser_start,\n                time.time(),\n                time.monotonic() - _t_ser_mono,\n            )\n            _t_grpc_start = time.time()\n            _t_grpc_mono = time.monotonic()\n\n        try:\n            response = self._stub.ModelInfer(\n                request,\n                timeout=self.timeout,\n                metadata=self._grpc_metadata,\n            )\n        except grpc.RpcError as exc:\n            raise RuntimeError(\n                f\"gRPC infer call failed for model {self.model_name}: {exc}\"\n            ) from exc\n\n        if _log.isEnabledFor(logging.DEBUG):\n            _log.debug(\n                \"PIPELINE_PROFILING KServe gRPC infer round-trip: batch_size=%d start=%.3f end=%.3f duration=%.3fs\",\n                _batch_size,\n                _t_grpc_start,\n                time.time(),\n                time.monotonic() - _t_grpc_mono,\n            )\n            _t_deser_start = time.time()\n            _t_deser_mono = time.monotonic()\n\n        decoded_outputs: Dict[str, np.ndarray] = {}\n\n        if self.use_binary_data:\n            if len(response.raw_output_contents) != len(response.outputs):","sourceCodeStart":329,"sourceCodeEnd":365,"githubUrl":"https://github.com/docling-project/docling/blob/61d76f1ff3f8428065465889f7b4577da7df704c/docling/models/inference_engines/common/kserve_v2_grpc.py#L329-L365","documentation":"The gRPC ModelInfer RPC raised grpc.RpcError and this RuntimeError is the chained wrapper that adds the model name. The underlying gRPC status code and details are in `exc` and the original error is accessible via __cause__. Typical codes: UNAVAILABLE (server down/unreachable), DEADLINE_EXCEEDED (timeout), NOT_FOUND (wrong model_name), PERMISSION_DENIED / UNAUTHENTICATED (missing credentials), RESOURCE_EXHAUSTED (server OOM or flow control).","triggerScenarios":"Wrong service URL/port for the gRPC endpoint; model_name not loaded on the KServe/Triton server; self.timeout too small for a big batch; TLS mismatch (client plaintext hitting TLS port); missing or expired auth metadata in _grpc_metadata; server restarting when the call lands.","commonSituations":"Cluster service renamed or port changed after redeploy; model not yet loaded when the first inference arrives; large batch inference exceeding a 5s default timeout; Istio/service-mesh sidecar rejecting the long-lived HTTP/2 stream; token in metadata expired.","solutions":["Read the nested grpc status: catch RuntimeError and inspect exc.__cause__.code() and .details() to get the real cause","For DEADLINE_EXCEEDED, raise the timeout value configured on the gRPC engine options","For UNAVAILABLE, verify the gRPC endpoint URL/port with grpcurl and check the server pod/process is up","For NOT_FOUND, confirm the model is loaded and that model_name/model_version match the server's model registry","For UNAUTHENTICATED/PERMISSION_DENIED, refresh or correct the metadata used for auth"],"exampleFix":"// before\noutputs = engine.infer(...)  # RuntimeError: gRPC infer call failed for model X: <AuroraError...>\n\n// after\ntry:\n    outputs = engine.infer(...)\nexcept RuntimeError as e:\n    rpc = e.__cause__\n    if rpc is not None and rpc.code() == grpc.StatusCode.DEADLINE_EXCEEDED:\n        ...  # retry with smaller batch","handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"import grpc\n\ntry:\n    outputs = engine.infer(inputs=inputs)\nexcept RuntimeError as e:\n    cause = e.__cause__\n    if isinstance(cause, grpc.RpcError):\n        code = cause.code()\n        if code in (grpc.StatusCode.UNAVAILABLE, grpc.StatusCode.DEADLINE_EXCEEDED):\n            outputs = engine.infer(inputs=inputs)  # or backoff-retry via tenacity\n        else:\n            raise\n    else:\n        raise","preventionTips":["Set a timeout sized for your largest batch, not your average one","Health-check the gRPC endpoint (channel_ready_future) before the first inference","Log e.__cause__.code()/.details() so failures are diagnosable","Keep model_name/model_version in sync with the server's loaded models"],"tags":["grpc","network","inference","timeout","runtime"],"backgroundTag":null,"analyzedSha":"61d76f1ff3f8428065465889f7b4577da7df704c","analyzedAt":"2026-08-14T23:53:18.727Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}