apache/beam · error · RuntimeError

Unknown error type

Error message

Unknown error type: {}

What it means

Raised in _assign_or_fail when the first element of a CUDA binding call's return tuple is not a cuda.CUresult instance, meaning the expected (err, ...) result shape from the cuda-python API was not received. It guards against unexpected return types before error-code checking.

Solutions

1. Pin/align the cuda-python package version with the version your Beam/TensorRT setup expects
2. Ensure _assign_or_fail is only invoked with raw tuples returned by cuda-python bindings (err first, then return values)
3. Upgrade the worker image's CUDA/cuda-python stack to a consistent, tested combination

Example fix

// before
_assign_or_fail(cuda.cuInit(-1))  # wrong call signature returns unexpected shape
// after
err, = cuda.cuInit(0)
_assign_or_fail((err,))

from cuda import cuda
def validate_cuda_binding_shape(result):
    if not (isinstance(result, tuple) and result and isinstance(result[0], cuda.CUresult)):
        raise RuntimeError('cuda-python call returned unexpected shape; check package version compatibility')

from cuda import cuda
def is_valid_cuda_result(args) -> bool:
    return isinstance(args, tuple) and len(args) >= 1 and isinstance(args[0], cuda.CUresult)

try:
    predictions = pcoll | RunInference(trt_handler)
except RuntimeError as e:
    if 'Unknown error type' in str(e):
        raise RuntimeError('cuda-python API returned unexpected result shape; align cuda-python version with the pipeline image') from e
    raise

When it happens

Trigger: A cuda-python API call returns an unexpected tuple shape/type (version drift between the code's expectations and the installed cuda-python package), and the result is passed to _assign_or_fail.

Common situations: Mismatched cuda-python (nvidia-cuda-runtime) package version in the worker image versus what tensorrt_inference expects; calling _assign_or_fail with a hand-built args tuple whose first element is not a CUresult; API changes across CUDA minor versions.

Related errors

• Cuda Error
• Failed to load ONNX file
• Failed to start vLLM server. Process status
• inference_args were provided, but should be None because…
• Unknown encoding

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/8cd27ef181635bcb. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/ml/inference/tensorrt_inference.py:94

def _build_engine(network, builder):
  import tensorrt as trt
  config = builder.create_builder_config()
  runtime = trt.Runtime(TRT_LOGGER)
  plan = builder.build_serialized_network(network, config)
  engine = runtime.deserialize_cuda_engine(plan)
  builder.reset()
  return engine


def _assign_or_fail(args):
  """CUDA error checking."""
  from cuda import cuda
  err, ret = args[0], args[1:]
  if isinstance(err, cuda.CUresult):
    if err != cuda.CUresult.CUDA_SUCCESS:
      raise RuntimeError("Cuda Error: {}".format(err))
  else:
    raise RuntimeError("Unknown error type: {}".format(err))
  # Special case so that no unpacking is needed at call-site.
  if len(ret) == 1:
    return ret[0]
  return ret


class TensorRTEngine:
  def __init__(self, engine: trt.ICudaEngine):
    """Implementation of the TensorRTEngine class which handles
    allocations associated with TensorRT engine.

    Example Usage::

      TensorRTEngine(engine)

    Args:
      engine: trt.ICudaEngine object that contains TensorRT engine
    """

View on GitHub (pinned to 12126d8942)