immich-app/immich · critical · RuntimeError

Failed to initialize RKNN runtime environment

Error message

Failed to initialize RKNN runtime environment

What it means

Raised by init_rknn() when RKNNLite.init_runtime() returns non-zero. load_rknn succeeded, so the model parses, but the NPU runtime could not be initialized. For SoCs in RKNN_COREMASK_SUPPORTED_SOCS the call passes core_mask=NPU_CORE_AUTO; otherwise it passes nothing. A non-zero return indicates a driver/device-level problem rather than a model problem.

Source

Thrown at machine-learning/immich_ml/sessions/rknn/rknnpool.py:56

    log.debug("RKNN is not available")


def init_rknn(model_path: str) -> "RKNNLite":
    if not is_available:
        raise RuntimeError("rknn is not available!")
    rknn_lite = RKNNLite()
    rknn_lite.rknn_log.logger.setLevel(logging.ERROR)
    ret = rknn_lite.load_rknn(model_path)
    if ret != 0:
        raise RuntimeError("Failed to load RKNN model")

    if soc_name in RKNN_COREMASK_SUPPORTED_SOCS:
        ret = rknn_lite.init_runtime(core_mask=RKNNLite.NPU_CORE_AUTO)
    else:
        ret = rknn_lite.init_runtime()  # Please do not set this parameter on other platforms.

    if ret != 0:
        raise RuntimeError("Failed to initialize RKNN runtime environment")

    return rknn_lite


class RknnPoolExecutor:
    def __init__(
        self,
        model_path: str,
        tpes: int,
        func: Callable[["RKNNLite", list[NDArray[np.float32]]], list[NDArray[np.float32]]],
    ) -> None:
        self.tpes = tpes
        self.queue: Queue[Future[list[NDArray[np.float32]]]] = Queue()
        self.rknn_pool = [init_rknn(model_path) for _ in range(tpes)]
        self.pool = ThreadPoolExecutor(max_workers=tpes)
        self.func = func
        self.num = 0

View on GitHub (pinned to 199723261c)

Solutions

1. Pass the NPU device into the container (e.g. `--device /dev/dri --device /dev/rknpu` or the compose equivalent) and grant read/write permission.
2. Confirm the rknpu kernel driver is loaded (`lsmod | grep rknpu`, `ls /dev/dri`) on the host.
3. Update the rknpu driver/firmware to a version that supports NPU_CORE_AUTO if your SoC is in RKNN_COREMASK_SUPPORTED_SOCS, or remove it from that set to use single-core init.
4. Reduce settings.rknn_threads so fewer concurrent RKNNLite runtimes contend for the NPU.
5. Ensure no other heavy NPU process is holding the device, then retry.

Example fix

# before
# container run without NPU passthrough -> init_runtime returns non-zero
session = RknnSession(Path('/models/model.rknn'))  # RuntimeError: Failed to initialize RKNN runtime environment

# after
# docker-compose.yml
services:
  immich-machine-learning:
    image: ghcr.io/immich-app/immich-machine-learning:rknn
    devices:
      - /dev/dri:/dev/dri
      - /dev/rknpu:/dev/rknpu
    group_add:
      - video
    environment:
      RKNN: true
session = RknnSession(Path('/models/model.rknn'))

from pathlib import Path

def validate_npu_device_accessible() -> None:
    required = [Path('/dev/dri'), Path('/dev/rknpu')]
    missing = [str(p) for p in required if not p.exists()]
    # /dev/rknpu may be optional on some kernels; require at least the dri render node
    render_nodes = list(Path('/dev/dri').glob('renderD*')) if Path('/dev/dri').exists() else []
    if not render_nodes:
        raise RuntimeError(
            f"No NPU render node found under /dev/dri; missing={missing}. "
            "Pass the device through to the container and load the rknpu driver."
        )

# call before constructing RknnSession:
validate_npu_device_accessible()

from pathlib import Path

def npu_device_present() -> bool:
    return Path('/dev/dri').exists() and any(Path('/dev/dri').glob('renderD*'))

try:
    session = RknnSession(model_path)
except RuntimeError as e:
    if 'Failed to initialize RKNN runtime' in str(e):
        log.error("NPU runtime init failed; check device passthrough and driver, then retry")
        validate_npu_device_accessible()
        session = RknnSession(model_path)  # single retry after fix
    else:
        raise

When it happens

Trigger: The NPU device node (/dev/dri/* or the rockchip NPU) is missing, busy, or lacks permissions; the kernel mali/rknpu driver is not loaded; the SoC is in RKNN_COREMASK_SUPPORTED_SOCS but multi-core NPU_CORE_AUTO is unsupported by the current driver; running inside a container without --device passthrough for the NPU; resource exhaustion from too many concurrent RKNNLite runtimes (one per pool thread).

Common situations: Container started without passing the NPU device through (`--device /dev/dri` or the rknpu device); rknpu kernel driver not loaded on the host; old driver that does not understand NPU_CORE_AUTO on an rk3588/rk3576; NPU already fully occupied by another process; permission denied on /dev/dri/renderD*.

Related errors

• rknn is not available!
• RKNN inference failed!
• Failed to load RKNN model
• Machine learning request '${JSON.stringify(config)}' failed
• libann is not available!

AI-assisted analysis of immich-app/immich@199723261c (2026-08-12). Data as JSON: /api/errors/2e8f87fa8e78f6bb. Report an issue: GitHub.