microsoft/semantic-kernel · error · ImportError

onnxruntime-genai is not installed.

Error message

onnxruntime-genai is not installed.

What it means

Raised as a plain ImportError (not a ServiceInitializationError) by the OnnxGenAICompletionBase constructor when the optional onnxruntime-genai package could not be imported at module load time. The module-level try/except sets ready=False on import failure, and __init__ checks this flag before proceeding. No model loading is attempted.

Source

Thrown at python/semantic_kernel/connectors/ai/onnx/services/onnx_gen_ai_completion_base.py:40

    """Base class for OnnxGenAI Completion services."""

    model: Any
    tokenizer: Any
    tokenizer_stream: Any
    enable_multi_modality: bool = False

    def __init__(self, ai_model_path: str, **kwargs) -> None:
        """Creates a new instance of the OnnxGenAICompletionBase class, loads model & tokenizer.

        Args:
            ai_model_path : Path to Onnx Model.
            **kwargs: Additional keyword arguments.

        Raises:
            ServiceInitializationError: When model cannot be loaded
        """
        if not ready:
            raise ImportError("onnxruntime-genai is not installed.")
        try:
            json_gen_ai_config = os.path.join(ai_model_path + "/genai_config.json")
            with open(json_gen_ai_config) as file:
                config: dict = json.load(file)
                enable_multi_modality = "vision" in config.get("model", {})
                model = OnnxRuntimeGenAi.Model(ai_model_path)
                if enable_multi_modality:
                    tokenizer = model.create_multimodal_processor()
                else:
                    tokenizer = OnnxRuntimeGenAi.Tokenizer(model)
                tokenizer_stream = tokenizer.create_stream()
        except Exception as ex:
            raise ServiceInitializationError("Failed to initialize OnnxCompletion service", ex) from ex

        super().__init__(
            model=model,
            tokenizer=tokenizer,
            tokenizer_stream=tokenizer_stream,

View on GitHub (pinned to c028a0c7dc)

Solutions

1. Install the ONNX runtime genai package: pip install onnxruntime-genai
2. Install semantic_kernel with the onnx extra: pip install semantic_kernel[onnx]
3. Verify the install: python -c 'import onnxruntime_genai'

Example fix

// before (missing dependency)
chat = OnnxGenAIChatCompletion(ai_model_path='/models/phi3')
// after
// pip install onnxruntime-genai
chat = OnnxGenAIChatCompletion(ai_model_path='/models/phi3')

try:
    import onnxruntime_genai  # noqa: F401
    ready = True
except ImportError:
    ready = False

if not ready:
    raise RuntimeError('onnxruntime-genai is not installed. Run: pip install onnxruntime-genai')

chat = OnnxGenAIChatCompletion(ai_model_path='/models/phi3')

try:
    chat = OnnxGenAIChatCompletion(ai_model_path=model_path)
except ImportError as e:
    if 'onnxruntime-genai' in str(e):
        logger.error('Install the dependency: pip install onnxruntime-genai')
    raise

When it happens

Trigger: Constructing OnnxGenAIChatCompletion or OnnxGenAITextCompletion in an environment where onnxruntime-genai is not installed. The __init__ method checks the module-level 'ready' flag and raises ImportError immediately.

Common situations: Installing semantic_kernel without the onnx extra (pip install semantic_kernel instead of pip install semantic_kernel[onnx]); fresh virtualenv; onnxruntime-genai not yet released for the platform/architecture (e.g. some ARM builds); package uninstalled during dependency cleanup.

Related errors

• Error creating OnnxGenAISettings: {e}
• AI model path is not provided. Please provide the 'ai_model_
• When using a multi-modal model, a template must be specified
• Failed to initialize OnnxCompletion service
• Invalid settings for OnnxGenAITextCompletion: {e}

AI-assisted analysis of microsoft/semantic-kernel@c028a0c7dc (2026-08-13). Data as JSON: /api/errors/8c7b5f7ce7e64001. Report an issue: GitHub.