Lightning-AI/pytorch-lightning · error · ValueError

Could not export to TensorRT since neither `input_sample` no

Error message

Could not export to TensorRT since neither `input_sample` nor `model.example_input_array` attribute is set.

What it means

Like to_onnx, TensorRT compilation needs a real input tensor to trace the model. If neither the `input_sample` argument nor `model.example_input_array` exists, Lightning raises ValueError.

Source

Thrown at src/lightning/pytorch/core/module.py:1667

                f"`{type(self).__name__}.to_tensorrt` requires `torch_tensorrt` to be installed. "
            )

        mode = self.training
        device = self.device
        if self.device.type != "cuda":
            default_device = torch.device(default_device) if isinstance(default_device, str) else default_device

            if not torch.cuda.is_available() or default_device.type != "cuda":
                raise MisconfigurationException(
                    f"TensorRT only supports CUDA devices. The current device is {self.device}."
                    f" Please set the `default_device` argument to a CUDA device."
                )

            self.to(default_device)

        if input_sample is None:
            if self.example_input_array is None:
                raise ValueError(
                    "Could not export to TensorRT since neither `input_sample` nor"
                    " `model.example_input_array` attribute is set."
                )
            input_sample = self.example_input_array

        import torch_tensorrt

        input_sample = copy.deepcopy((input_sample,) if isinstance(input_sample, torch.Tensor) else input_sample)
        input_sample = self._on_before_batch_transfer(input_sample)
        input_sample = self._apply_batch_transfer_handler(input_sample)

        with _jit_is_scripting() if ir == "ts" else nullcontext():
            trt_obj = torch_tensorrt.compile(
                module=self.eval(),
                ir=ir,
                inputs=input_sample,
                **compile_kwargs,
            )

View on GitHub (pinned to 9fed5c27d2)

Solutions

1. Pass input_sample=torch.randn(1, *input_shape) (on the right device)
2. Or set self.example_input_array in the model's __init__

Example fix

# before
model.to_tensorrt("model.ep")
# after
model.to_tensorrt("model.ep", input_sample=torch.randn(1, 3, 224, 224, device="cuda"))

sample = input_sample if input_sample is not None else getattr(model, "example_input_array", None)
if sample is None:
    raise ValueError("input required for TensorRT export")
model.to_tensorrt(path, input_sample=sample)

When it happens

Trigger: Calling `model.to_tensorrt("model.ep")` without input_sample on a model lacking example_input_array.

Common situations: Exporting a newly written LightningModule for inference deployment without having set an example input.

Understand the failure class

Background: "missing required argument" and "the following required arguments were not provided": what required-argument errors mean and how to fix them — this error's family across 20 libraries.

Related errors

• Could not export to ONNX since neither `input_sample` nor `m
• Choosing method=`trace` requires either `example_inputs` or
• `{type(self).__name__}.to_tensorrt` requires `torch_tensorrt
• TensorRT only supports CUDA devices. The current device is {
• TensorRT with IR mode 'ts' only supports output format 'torc

AI-assisted analysis of Lightning-AI/pytorch-lightning@9fed5c27d2 (2026-08-28). Data as JSON: /api/errors/fe438fdc92582702. Report an issue: GitHub.