Lightning-AI/pytorch-lightning · error · MisconfigurationException

TensorRT only supports CUDA devices. The current device is {

Error message

TensorRT only supports CUDA devices. The current device is {self.device}. Please set the `default_device` argument to a CUDA device.

What it means

TensorRT engines can only be built for CUDA devices. If the model currently sits on a non-CUDA device (e.g. cpu) and either CUDA is unavailable or the provided default_device is not a CUDA device, Lightning raises MisconfigurationException telling you to set `default_device` to a CUDA device.

Source

Thrown at src/lightning/pytorch/core/module.py:1658

            input_sample = torch.randn(1, 64)
            exported_program = model.to_tensorrt(
                file_path="export.ep",
                inputs=input_sample,
            )

        """
        if not _TORCH_TRT_AVAILABLE:
            raise ModuleNotFoundError(
                f"`{type(self).__name__}.to_tensorrt` requires `torch_tensorrt` to be installed. "
            )

        mode = self.training
        device = self.device
        if self.device.type != "cuda":
            default_device = torch.device(default_device) if isinstance(default_device, str) else default_device

            if not torch.cuda.is_available() or default_device.type != "cuda":
                raise MisconfigurationException(
                    f"TensorRT only supports CUDA devices. The current device is {self.device}."
                    f" Please set the `default_device` argument to a CUDA device."
                )

            self.to(default_device)

        if input_sample is None:
            if self.example_input_array is None:
                raise ValueError(
                    "Could not export to TensorRT since neither `input_sample` nor"
                    " `model.example_input_array` attribute is set."
                )
            input_sample = self.example_input_array

        import torch_tensorrt

        input_sample = copy.deepcopy((input_sample,) if isinstance(input_sample, torch.Tensor) else input_sample)
        input_sample = self._on_before_batch_transfer(input_sample)

View on GitHub (pinned to 9fed5c27d2)

Solutions

1. Run on a GPU machine and pass default_device="cuda:0" (or move the model to cuda first)
2. Verify torch.cuda.is_available() before attempting export
3. If no GPU is available, use to_onnx() instead and compile with TensorRT offline

Example fix

# before
model.to_tensorrt("model.ep", input_sample=x)  # model on cpu
# after
model.to_tensorrt("model.ep", input_sample=x, default_device="cuda:0")

import torch
if not torch.cuda.is_available():
    raise RuntimeError("TensorRT export requires CUDA")
model.to_tensorrt(path, input_sample=x, default_device="cuda:0")

When it happens

Trigger: Calling model.to_tensorrt() with the model on CPU and default_device unset/"cpu", or on a machine without CUDA.

Common situations: Running the export script on a CPU-only dev box or CI node, or forgetting to move the model before export.

Related errors

• CPU parallel_devices set through {self._strategy_flag.__clas
• GPU parallel_devices set through {self._strategy_flag.__clas
• At least one gpu type should be specified!
• You requested gpu: {gpus} But your machine only has: {all_a
• `{type(self).__name__}.to_tensorrt` requires `torch_tensorrt

AI-assisted analysis of Lightning-AI/pytorch-lightning@9fed5c27d2 (2026-08-28). Data as JSON: /api/errors/101dff76a2dfe2e3. Report an issue: GitHub.