What causes "Failed to initialize OnnxCompletion service" in microsoft/semantic-kernel?

Inspect the chained exception (__cause__) for the specific failure (FileNotFoundError, JSONDecodeError, etc.)

microsoft/semantic-kernel · error · ServiceInitializationError

Failed to initialize OnnxCompletion service

Error message

Failed to initialize OnnxCompletion service

What it means

Raised by the OnnxGenAICompletionBase constructor when any Exception occurs during model/tokenizer loading — reading genai_config.json, instantiating OnnxRuntimeGenAi.Model, or creating the tokenizer/stream. The broad except clause wraps ALL exceptions as ServiceInitializationError with the original exception chained via 'from ex'. The most common cause is an invalid or missing model path.

Solutions

Inspect the chained exception (__cause__) for the specific failure (FileNotFoundError, JSONDecodeError, etc.)
Verify ai_model_path points to a folder containing genai_config.json and model .onnx files
Ensure the onnxruntime-genai version is compatible with the model format
Check file read permissions on the model folder

Defensive patterns

Strategy: try-catch

Validate before calling

import os, json

model_path = '/path/to/model_folder'
config_path = os.path.join(model_path, 'genai_config.json')

if not os.path.isdir(model_path):
    raise RuntimeError(f'Model folder does not exist: {model_path}')
if not os.path.isfile(config_path):
    raise RuntimeError(f'genai_config.json not found in: {model_path}')
try:
    with open(config_path) as f:
        json.load(f)
except json.JSONDecodeError:
    raise RuntimeError(f'genai_config.json is not valid JSON')

Try / catch

from semantic_kernel.exceptions.service_exceptions import ServiceInitializationError

try:
    chat = OnnxGenAIChatCompletion(ai_model_path=model_path)
except ServiceInitializationError as e:
    cause = e.__cause__
    logger.error('ONNX init failed: %s. Root cause: %s', e, cause)
    raise

Prevention

Verify the model folder exists and contains genai_config.json before constructing
Download models from a trusted source to avoid corrupt or partial folders
Pin a compatible onnxruntime-genai version for your model format
Log e.__cause__ to diagnose the underlying file/parse error

When it happens

Trigger: Constructing the service with an ai_model_path that does not exist, lacks genai_config.json, has a corrupt config JSON, or where the model files are incompatible with the installed onnxruntime-genai version. Also raised on file permission errors.

Common situations: Pointing at a partially downloaded model folder; wrong path (missing trailing directory, relative vs absolute); model config format mismatch with onnxruntime-genai version; file permissions preventing read; genai_config.json missing because the model was downloaded via a different tool.

Related errors

AI-assisted analysis of microsoft/semantic-kernel@c028a0c7dc (2026-08-13). Data as JSON: /api/errors/e731f8eeb20c9dc1. Report an issue: GitHub.

Appendix: source

Thrown at python/semantic_kernel/connectors/ai/onnx/services/onnx_gen_ai_completion_base.py:53

        Raises:
            ServiceInitializationError: When model cannot be loaded
        """
        if not ready:
            raise ImportError("onnxruntime-genai is not installed.")
        try:
            json_gen_ai_config = os.path.join(ai_model_path + "/genai_config.json")
            with open(json_gen_ai_config) as file:
                config: dict = json.load(file)
                enable_multi_modality = "vision" in config.get("model", {})
                model = OnnxRuntimeGenAi.Model(ai_model_path)
                if enable_multi_modality:
                    tokenizer = model.create_multimodal_processor()
                else:
                    tokenizer = OnnxRuntimeGenAi.Tokenizer(model)
                tokenizer_stream = tokenizer.create_stream()
        except Exception as ex:
            raise ServiceInitializationError("Failed to initialize OnnxCompletion service", ex) from ex

        super().__init__(
            model=model,
            tokenizer=tokenizer,
            tokenizer_stream=tokenizer_stream,
            enable_multi_modality=enable_multi_modality,
            **kwargs,
        )

    async def _generate_next_token_async(
        self,
        prompt: str,
        settings: OnnxGenAIPromptExecutionSettings,
        images: list[ImageContent] | None = None,
        audios: list[AudioContent] | None = None,
    ) -> AsyncGenerator[list[str], Any]:
        try:
            params = OnnxRuntimeGenAi.GeneratorParams(self.model)

View on GitHub (pinned to c028a0c7dc)