apache/beam · error · ValueError

inference_args were provided, but should be None because…

Error message

inference_args were provided, but should be None because this framework does not expect extra arguments on inferences.

What it means

TensorRT model handlers in Apache Beam do not accept extra per-request arguments. validate_inference_args raises ValueError whenever a non-empty inference_args dict is passed, because the TensorRT runtime has no parameters to forward at inference time.

Solutions

1. Remove the inference_args argument (or pass None) from the RunInference/inference call
2. Bake the would-be inference arguments into the model config or handler constructor (e.g. via tensorrt_dims/engine settings) instead of per-request
3. If you need per-request args, switch to a model handler that supports inference_args (e.g. PyTorch or ONNX handlers)

Example fix

// before
result = beam.RunInference(tensorrt_handler, inference_args={'image_size': 512})
// after
result = beam.RunInference(tensorrt_handler, inference_args=None)

if inference_args is not None and len(inference_args) > 0:
    raise TypeError('TensorRT handler does not accept inference_args; pass None')
result = beam.RunInference(tensorrt_handler, inference_args=None)

def no_inference_args(args) -> bool:
    return args is None or len(args) == 0

When it happens

Trigger: Calling RunInference (or model_handler.predict/inference) with inference_args set to any non-empty dict, e.g. inference_args={'image_size': 512}, against a TensorRTModelHandler.

Common situations: Reusing pipeline code written for ONNX/PyTorch/sklearn handlers where inference_args like top_k or image size are supported; copying examples from another framework's docs.

Understand the failure class

Background: UnsupportedOperationException and "is not supported" errors: when a library deliberately refuses a call — this error's family across 30 libraries.

Related errors

• Cuda Error
• Failed to load ONNX file
• Basepath %r must be GCS path.
• Callable create_model_fn must be passedwith…
• Cannot make make an unkeyed model handler with pre or…

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/09d5c2749876cf48. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/ml/inference/tensorrt_inference.py:352

    """
    Returns:
      The number of bytes of data for a batch of Tensors.
    """
    return sum((np_array.itemsize for np_array in batch))

  def get_metrics_namespace(self) -> str:
    """
    Returns a namespace for metrics collected by the RunInference transform.
    """
    return 'BeamML_TensorRT'

  def validate_inference_args(self, inference_args: Optional[dict[str, Any]]):
    """
    Currently, this model handler does not support inference args. Given that,
    we will throw if any are passed in.
    """
    if inference_args:
      raise ValueError(
          'inference_args were provided, but should be None because this '
          'framework does not expect extra arguments on inferences.')

View on GitHub (pinned to 12126d8942)