{"record":{"id":"f3a39ade6fd38ebd","repo":"ultralytics/yolov5","slug":"tensorrt-engine-deserialization-failed-re-export","errorCode":null,"errorMessage":"TensorRT engine deserialization failed. Re-export the engine with the same TensorRT version, CUDA version, and GPU device used for inference.","messagePattern":"TensorRT engine deserialization failed\\. Re-export the engine with the same TensorRT version, CUDA version, and GPU device used for inference\\.","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"models/common.py","lineNumber":545,"sourceCode":"                ov_model.get_parameters()[0].set_layout(Layout(\"NCHW\"))\n            batch_dim = get_batch(ov_model)\n            if batch_dim.is_static:\n                batch_size = batch_dim.get_length()\n            ov_compiled_model = core.compile_model(ov_model, device_name=\"AUTO\")  # AUTO selects best available device\n            stride, names = self._load_metadata(Path(w).with_suffix(\".yaml\"))  # load metadata\n        elif engine:  # TensorRT\n            LOGGER.info(f\"Loading {w} for TensorRT inference...\")\n            import tensorrt as trt  # https://developer.nvidia.com/nvidia-tensorrt-download\n\n            check_version(trt.__version__, \"7.0.0\", hard=True)  # require tensorrt>=7.0.0\n            if device.type == \"cpu\":\n                device = torch.device(\"cuda:0\")\n            Binding = namedtuple(\"Binding\", (\"name\", \"dtype\", \"shape\", \"data\", \"ptr\"))\n            logger = trt.Logger(trt.Logger.INFO)\n            with open(w, \"rb\") as f, trt.Runtime(logger) as runtime:\n                model = runtime.deserialize_cuda_engine(f.read())\n            if model is None:\n                raise RuntimeError(\n                    \"TensorRT engine deserialization failed. Re-export the engine with the same TensorRT version, \"\n                    \"CUDA version, and GPU device used for inference.\"\n                )\n            context = model.create_execution_context()\n            bindings = OrderedDict()\n            output_names = []\n            fp16 = False  # default updated below\n            dynamic = False\n            is_trt10 = not hasattr(model, \"num_bindings\")\n            num = range(model.num_io_tensors) if is_trt10 else range(model.num_bindings)\n            for i in num:\n                if is_trt10:\n                    name = model.get_tensor_name(i)\n                    dtype = trt.nptype(model.get_tensor_dtype(name))\n                    is_input = model.get_tensor_mode(name) == trt.TensorIOMode.INPUT\n                    if is_input:\n                        if -1 in tuple(model.get_tensor_shape(name)):  # dynamic\n                            dynamic = True","sourceCodeStart":527,"sourceCodeEnd":563,"githubUrl":"https://github.com/ultralytics/yolov5/blob/20d1d78a08277e365d57bfa3a2cce752772d9e59/models/common.py#L527-L563","documentation":"DetectMultiBackend raises RuntimeError when TensorRT's deserialize_cuda_engine returns None while loading a .engine file. TensorRT engines are not portable: they are tied to the exact TensorRT version, CUDA version, and GPU architecture they were built with, and deserialization silently returns None on mismatch rather than raising inside TensorRT.","triggerScenarios":"Passing an .engine file exported on another machine to DetectMultiBackend; running an engine built with TensorRT 8.x under TensorRT 10.x; an engine built for a different GPU compute capability (e.g. exported on A100, run on T4); a truncated engine file from a partial copy.","commonSituations":"Shipping .engine files in Docker images without pinning the TensorRT base image; upgrading the tensorrt pip package (or the NGC container tag) without re-exporting; deploying the same artifact fleet-wide across heterogeneous GPUs.","solutions":["Re-export the engine on the exact machine/GPU and TensorRT version used for inference: python export.py --weights yolov5s.pt --include engine --device 0.","Align versions end-to-end: same tensorrt pip package, same CUDA, same GPU model between export and inference.","If you must ship one artifact, ship the .onnx or .pt and build the engine as a deployment step on the target host.","Verify integrity of the engine file (size, sha256) if it was copied between hosts."],"exampleFix":"# before (engine exported elsewhere)\nmodel = DetectMultiBackend('yolov5s.engine', device=torch.device('cuda:0'))\n\n# after (build on the inference host, then load)\n# python export.py --weights yolov5s.pt --include engine --device 0 --half\nmodel = DetectMultiBackend('yolov5s.engine', device=torch.device('cuda:0'))","handlingStrategy":"validation","validationCode":"import tensorrt as trt\n\ndef engine_parses(path: str) -> bool:\n    logger = trt.Logger(trt.Logger.ERROR)\n    with open(path, 'rb') as f, trt.Runtime(logger) as rt:\n        return rt.deserialize_cuda_engine(f.read()) is not None","typeGuard":null,"tryCatchPattern":"try:\n    model = DetectMultiBackend('yolov5s.engine', device=device)\nexcept RuntimeError as e:\n    if 'deserialization failed' in str(e):\n        subprocess.run(['python', 'export.py', '--weights', 'yolov5s.pt', '--include', 'engine'], check=True)\n        model = DetectMultiBackend('yolov5s.engine', device=device)","preventionTips":["Build engines on the target host as a deployment step, never ship them across GPU types.","Pin the TensorRT/CUDA versions of export and inference containers to identical tags.","Log trt.__version__ and GPU name at both export and inference time for auditability."],"tags":["tensorrt","inference","version-mismatch","deployment"],"backgroundTag":null,"analyzedSha":"20d1d78a08277e365d57bfa3a2cce752772d9e59","analyzedAt":"2026-08-15T02:56:15.443Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}