immich-app/immich · critical · RuntimeError

libann is not available!

Error message

libann is not available!

What it means

Raised by Ann.__init__ when the module-level is_available flag is False. That flag is set False at import time if CDLL('libmali.so') or CDLL('libann.so') raised OSError — i.e. the ARM Mali userspace driver (libmali.so) and/or the Arm NN wrapper library (libann.so) are not present/loadable in the container or host. Ann is a singleton, so this fires on the first Ann() construction (which AnnSession does for .armnn models).

Source

Thrown at machine-learning/immich_ml/sessions/ann/loader.py:54

class _Singleton(type, Newable[T]):
    _instances: dict[_Singleton[T], Newable[T]] = {}

    def __call__(cls, *args: Any, **kwargs: Any) -> Newable[T]:
        if cls not in cls._instances:
            obj: Newable[T] = super(_Singleton, cls).__call__(*args, **kwargs)
            cls._instances[cls] = obj
        else:
            obj = cls._instances[cls]
            obj.new()
        return obj


class Ann(metaclass=_Singleton):
    def __init__(self, log_level: int = 3, tuning_level: int = 1, tuning_file: str | None = None) -> None:
        if not is_available:
            raise RuntimeError("libann is not available!")
        if tuning_level == 0 and tuning_file is None:
            raise ValueError("tuning_level == 0 reads existing tuning information and requires a tuning_file")
        if tuning_level < 0 or tuning_level > 3:
            raise ValueError("tuning_level must be 0 (load from tuning_file), 1, 2 or 3.")
        if log_level < 0 or log_level > 5:
            raise ValueError("log_level must be 0 (trace), 1 (debug), 2 (info), 3 (warning), 4 (error) or 5 (fatal)")
        self.log_level = log_level
        self.tuning_level = tuning_level
        self.tuning_file = tuning_file
        self.output_shapes: dict[int, tuple[tuple[int], ...]] = {}
        self.input_shapes: dict[int, tuple[tuple[int], ...]] = {}
        self.ann: int | None = None
        self.new()

        if self.tuning_file is not None:
            # make sure tuning file exists (without clearing contents)
            # once filled, the tuning file reduces the cost/time of the first
            # inference after model load by 10s of seconds

View on GitHub (pinned to 199723261c)

Solutions

1. Confirm you actually intend to use ARMNN: switch model_format to ONNX (or RKNN on Rockchip) unless you are on a Mali GPU platform.
2. Use the immich-ml ARMNN-tagged image and ensure libmali.so is mounted from the host into /usr/lib if required by that image.
3. Run `ldd /usr/lib/libann.so` and `ldd /usr/lib/libmali.so` in the container and install/fix every 'not found' dependency.
4. Set LD_LIBRARY_PATH (or configure the runtime) to the directory containing both libraries and restart the service.
5. Check the container's debug log for the original OSError line ('Could not load ANN shared libraries, using ONNX') which names the missing library.

Example fix

# before
# x86 host, ARMNN image without libs -> is_available=False
session = AnnSession(model_path)   # RuntimeError: libann is not available!

# after
# docker-compose.yml
services:
  immich-machine-learning:
    image: ghcr.io/immich-app/immich-machine-learning:cuda
    # use the right variant; or mount mali on ARM:
    #   volumes:
    #     - /usr/lib/libmali.so:/usr/lib/libmali.so:ro
model = InferenceModel('immich-app/X', model_format=ModelFormat.ONNX)

from immich_ml.sessions.ann.loader import is_available

def ensure_armnn_available() -> None:
    if not is_available:
        raise RuntimeError(
            "libann/libmali not loadable; use the ARMNN image variant with libmali.so mounted, "
            "or switch model_format to ONNX. See the 'Could not load ANN shared libraries' debug line."
        )

# call before constructing an AnnSession:
ensure_armnn_available()

from immich_ml.sessions.ann.loader import is_available

def armnn_runtime_available() -> bool:
    return bool(is_available)

from immich_ml.sessions.ann.loader import is_available
from immich_ml.sessions.ort import OrtSession

try:
    session = AnnSession(model_path)
except RuntimeError as e:
    if not is_available:
        log.warning("ARMNN unavailable, falling back to ONNX session")
        session = OrtSession(model_path.with_suffix('.onnx'))
    else:
        raise

When it happens

Trigger: Constructing an AnnSession / loading a model with model_format=ARMNN on a host that is not an ARM Mali GPU device, or in a container image built without libann.so and libmali.so mounted. Also triggered if the libraries exist but have unresolved dependencies (wrong glibc, missing libstdc++), which CDLL surfaces as OSError.

Common situations: Running the default (non-ARM) immich-ml image on x86_64; forgetting to mount the vendor libmali.so blob into the container; LD_LIBRARY_PATH not including the directory holding libann.so; ABI mismatch between libmali.so and the host kernel mali driver; using the ARMNN image on a Rockchip board that should use RKNN instead.

Related errors

• Cannot load model!
• rknn is not available!
• No /dev/dri devices found. If using Docker, make sure at lea
• Device '${deviceName}' does not exist. If using Docker, make
• model_path must be a file with extension .armnn, .tflite or

AI-assisted analysis of immich-app/immich@199723261c (2026-08-12). Data as JSON: /api/errors/acd11e533bd2ba65. Report an issue: GitHub.