{"record":{"id":"63f8518552cccfc9","repo":"apache/beam","slug":"cuda-error","errorCode":null,"errorMessage":"Cuda Error: {}","messagePattern":"Cuda Error: (.+?)","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/ml/inference/tensorrt_inference.py","lineNumber":92,"sourceCode":"\n\ndef _build_engine(network, builder):\n  import tensorrt as trt\n  config = builder.create_builder_config()\n  runtime = trt.Runtime(TRT_LOGGER)\n  plan = builder.build_serialized_network(network, config)\n  engine = runtime.deserialize_cuda_engine(plan)\n  builder.reset()\n  return engine\n\n\ndef _assign_or_fail(args):\n  \"\"\"CUDA error checking.\"\"\"\n  from cuda import cuda\n  err, ret = args[0], args[1:]\n  if isinstance(err, cuda.CUresult):\n    if err != cuda.CUresult.CUDA_SUCCESS:\n      raise RuntimeError(\"Cuda Error: {}\".format(err))\n  else:\n    raise RuntimeError(\"Unknown error type: {}\".format(err))\n  # Special case so that no unpacking is needed at call-site.\n  if len(ret) == 1:\n    return ret[0]\n  return ret\n\n\nclass TensorRTEngine:\n  def __init__(self, engine: trt.ICudaEngine):\n    \"\"\"Implementation of the TensorRTEngine class which handles\n    allocations associated with TensorRT engine.\n\n    Example Usage::\n\n      TensorRTEngine(engine)\n\n    Args:","sourceCodeStart":74,"sourceCodeEnd":110,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/ml/inference/tensorrt_inference.py#L74-L110","documentation":"_assign_or_fail performs CUDA error checking on results returned by cuda-python bindings. If the returned CUresult code is not CUDA_SUCCESS, it raises RuntimeError('Cuda Error: {code}') with the numeric/enum error value.","triggerScenarios":"Any CUDA API call routed through _assign_or_fail (during TensorRT engine setup in __init__ or inference in _default_tensorRT_inference_fn) returning a non-success status, e.g. CUDA_ERROR_INVALID_VALUE, CUDA_ERROR_OUT_OF_MEMORY, or CUDA_ERROR_NO_DEVICE.","commonSituations":"Worker machines without a visible GPU or with the wrong CUDA driver; requesting an engine/device config exceeding GPU memory; running the pipeline on a CPU-only runner while the handler requires a GPU; CUDA driver/runtime version mismatch in the container.","solutions":["Decode the reported CUresult code to identify the specific failure (e.g. via cuda.CUresult(code).name)","Verify workers are GPU instances with a matching CUDA driver and that the image's CUDA/tensorrt versions are compatible","Check the model/engine size fits the GPU memory budget and adjust builder config (max workspace, precision) accordingly","Confirm the runner/scheduler exposes GPUs and CUDA_VISIBLE_DEVICES is not hiding devices"],"exampleFix":"// before\nhandler = TensorRTEngineHandlerNumpy(..., precision_mode=bf16)  # GPU not visible on worker\n// after\n# run on GPU worker pool, or pick precision supported by the device\nhandler = TensorRTEngineHandlerNumpy(..., precision_mode=fp16)","handlingStrategy":"try-catch","validationCode":"from cuda import cuda\ndef check_cuda_available():\n    err, = cuda.cuInit(0)\n    if err != cuda.CUresult.CUDA_SUCCESS:\n        raise RuntimeError(f'CUDA init failed: {cuda.CUresult(err).name}')","typeGuard":"from cuda import cuda\ndef is_cuda_success(result) -> bool:\n    err = result[0]\n    return isinstance(err, cuda.CUresult) and err == cuda.CUresult.CUDA_SUCCESS","tryCatchPattern":"try:\n    predictions = pcoll | RunInference(trt_handler)\nexcept RuntimeError as e:\n    if str(e).startswith('Cuda Error'):\n        raise RuntimeError('Check GPU availability, driver/CUDA version match, and memory limits on workers') from e\n    raise","preventionTips":["Verify worker machines expose GPUs and the CUDA driver matches the image's CUDA toolkit","Keep engine/workspace sizes within GPU memory; reduce precision (fp16/int8) if needed","Smoke-test CUDA initialization in the exact worker container before running pipelines"],"tags":["cuda","tensorrt","gpu","apache-beam","ml-inference"],"backgroundTag":"cuda-error","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}