{"record":{"id":"2e8f87fa8e78f6bb","repo":"immich-app/immich","slug":"failed-to-initialize-rknn-runtime-environment","errorCode":null,"errorMessage":"Failed to initialize RKNN runtime environment","messagePattern":"Failed to initialize RKNN runtime environment","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"critical","filePath":"machine-learning/immich_ml/sessions/rknn/rknnpool.py","lineNumber":56,"sourceCode":"    log.debug(\"RKNN is not available\")\n\n\ndef init_rknn(model_path: str) -> \"RKNNLite\":\n    if not is_available:\n        raise RuntimeError(\"rknn is not available!\")\n    rknn_lite = RKNNLite()\n    rknn_lite.rknn_log.logger.setLevel(logging.ERROR)\n    ret = rknn_lite.load_rknn(model_path)\n    if ret != 0:\n        raise RuntimeError(\"Failed to load RKNN model\")\n\n    if soc_name in RKNN_COREMASK_SUPPORTED_SOCS:\n        ret = rknn_lite.init_runtime(core_mask=RKNNLite.NPU_CORE_AUTO)\n    else:\n        ret = rknn_lite.init_runtime()  # Please do not set this parameter on other platforms.\n\n    if ret != 0:\n        raise RuntimeError(\"Failed to initialize RKNN runtime environment\")\n\n    return rknn_lite\n\n\nclass RknnPoolExecutor:\n    def __init__(\n        self,\n        model_path: str,\n        tpes: int,\n        func: Callable[[\"RKNNLite\", list[NDArray[np.float32]]], list[NDArray[np.float32]]],\n    ) -> None:\n        self.tpes = tpes\n        self.queue: Queue[Future[list[NDArray[np.float32]]]] = Queue()\n        self.rknn_pool = [init_rknn(model_path) for _ in range(tpes)]\n        self.pool = ThreadPoolExecutor(max_workers=tpes)\n        self.func = func\n        self.num = 0\n","sourceCodeStart":38,"sourceCodeEnd":74,"githubUrl":"https://github.com/immich-app/immich/blob/199723261c6ffa897fec8ccdaea6359e39c37cc3/machine-learning/immich_ml/sessions/rknn/rknnpool.py#L38-L74","documentation":"Raised by init_rknn() when RKNNLite.init_runtime() returns non-zero. load_rknn succeeded, so the model parses, but the NPU runtime could not be initialized. For SoCs in RKNN_COREMASK_SUPPORTED_SOCS the call passes core_mask=NPU_CORE_AUTO; otherwise it passes nothing. A non-zero return indicates a driver/device-level problem rather than a model problem.","triggerScenarios":"The NPU device node (/dev/dri/* or the rockchip NPU) is missing, busy, or lacks permissions; the kernel mali/rknpu driver is not loaded; the SoC is in RKNN_COREMASK_SUPPORTED_SOCS but multi-core NPU_CORE_AUTO is unsupported by the current driver; running inside a container without --device passthrough for the NPU; resource exhaustion from too many concurrent RKNNLite runtimes (one per pool thread).","commonSituations":"Container started without passing the NPU device through (`--device /dev/dri` or the rknpu device); rknpu kernel driver not loaded on the host; old driver that does not understand NPU_CORE_AUTO on an rk3588/rk3576; NPU already fully occupied by another process; permission denied on /dev/dri/renderD*.","solutions":["Pass the NPU device into the container (e.g. `--device /dev/dri --device /dev/rknpu` or the compose equivalent) and grant read/write permission.","Confirm the rknpu kernel driver is loaded (`lsmod | grep rknpu`, `ls /dev/dri`) on the host.","Update the rknpu driver/firmware to a version that supports NPU_CORE_AUTO if your SoC is in RKNN_COREMASK_SUPPORTED_SOCS, or remove it from that set to use single-core init.","Reduce settings.rknn_threads so fewer concurrent RKNNLite runtimes contend for the NPU.","Ensure no other heavy NPU process is holding the device, then retry."],"exampleFix":"# before\n# container run without NPU passthrough -> init_runtime returns non-zero\nsession = RknnSession(Path('/models/model.rknn'))  # RuntimeError: Failed to initialize RKNN runtime environment\n\n# after\n# docker-compose.yml\nservices:\n  immich-machine-learning:\n    image: ghcr.io/immich-app/immich-machine-learning:rknn\n    devices:\n      - /dev/dri:/dev/dri\n      - /dev/rknpu:/dev/rknpu\n    group_add:\n      - video\n    environment:\n      RKNN: true\nsession = RknnSession(Path('/models/model.rknn'))","handlingStrategy":"validation","validationCode":"from pathlib import Path\n\ndef validate_npu_device_accessible() -> None:\n    required = [Path('/dev/dri'), Path('/dev/rknpu')]\n    missing = [str(p) for p in required if not p.exists()]\n    # /dev/rknpu may be optional on some kernels; require at least the dri render node\n    render_nodes = list(Path('/dev/dri').glob('renderD*')) if Path('/dev/dri').exists() else []\n    if not render_nodes:\n        raise RuntimeError(\n            f\"No NPU render node found under /dev/dri; missing={missing}. \"\n            \"Pass the device through to the container and load the rknpu driver.\"\n        )\n\n# call before constructing RknnSession:\nvalidate_npu_device_accessible()","typeGuard":"from pathlib import Path\n\ndef npu_device_present() -> bool:\n    return Path('/dev/dri').exists() and any(Path('/dev/dri').glob('renderD*'))","tryCatchPattern":"try:\n    session = RknnSession(model_path)\nexcept RuntimeError as e:\n    if 'Failed to initialize RKNN runtime' in str(e):\n        log.error(\"NPU runtime init failed; check device passthrough and driver, then retry\")\n        validate_npu_device_accessible()\n        session = RknnSession(model_path)  # single retry after fix\n    else:\n        raise","preventionTips":["Always pass `/dev/dri` (and `/dev/rknpu` where present) into the RKNN container and add the `video` group.","Ensure the rknpu kernel driver is loaded on the host before starting the service.","Match RKNN_COREMASK_SUPPORTED_SOCS to driver capability; do not request NPU_CORE_AUTO on old drivers.","Limit settings.rknn_threads to physical NPU core count to avoid init-time resource exhaustion."],"tags":["rknn","npu","runtime","rockchip","device-permissions","container"],"backgroundTag":null,"analyzedSha":"199723261c6ffa897fec8ccdaea6359e39c37cc3","analyzedAt":"2026-08-12T04:54:27.085Z","schemaVersion":2},"datasetVersion":"2026-08-12T13:17:24.610Z"}