apache/beam · error · UserCodeExecutionException

could not complete request

Error message

could not complete request

What it means

_execute_request converts any RuntimeError raised by the user callable into UserCodeExecutionException with the message 'could not complete request'. It signals that the request could not be executed, separate from HTTP error codes or timeouts.

Source

Thrown at sdks/python/apache_beam/io/requestresponse.py:211

  with concurrent.futures.ThreadPoolExecutor() as executor:
    future = executor.submit(caller, request)
    try:
      return future.result(timeout=timeout)
    except TooManyRequests as e:
      _LOGGER.info(
          'request could not be completed. got code %i from the service.',
          e.code)
      raise e
    except concurrent.futures.TimeoutError:
      if metrics_collector:
        metrics_collector.timeout_requests.inc(1)
      raise UserCodeTimeoutException(
          f'Timeout {timeout} exceeded '
          f'while completing request: {request}')
    except RuntimeError:
      if metrics_collector:
        metrics_collector.failures.inc(1)
      raise UserCodeExecutionException('could not complete request')


class ExponentialBackOffRepeater(Repeater):
  """Configure exponential backoff retry strategy.

  It retries for exceptions due to the remote service such as
  TooManyRequests (HTTP 429), UserCodeTimeoutException, UserCodeQuotaException.

  It utilizes the decorator
  :func:`apache_beam.utils.retry.with_exponential_backoff`.
  """
  def __init__(self):
    pass

  @retry.with_exponential_backoff(
      num_retries=2, retry_filter=retry_on_exception)
  def repeat(
      self,

View on GitHub (pinned to 12126d8942)

Solutions

1. Inspect the original exception in the logs/cause to find the root cause inside your Caller implementation.
2. Fix the user callable so it does not raise RuntimeError (e.g. handle connection errors and raise UserCodeExecutionException with details yourself).
3. Add retry logic (Repeater) for transient remote service failures.
4. Add defensive error handling and validation of inputs/connections in the callable.

Example fix

// before
def __call__(self, request):
  return requests.post(self.url, json=request).json()  # may raise RuntimeError
// after
def __call__(self, request):
  try:
    return requests.post(self.url, json=request, timeout=10).json()
  except Exception as e:
    raise UserCodeExecutionException(f'request failed: {e}') from e

try:
  result = do_request(request)
except UserCodeExecutionException as e:
  logging.error('request failed: %s', e)
  # inspect root cause via e.__cause__ and retry or dead-letter

When it happens

Trigger: The user-supplied callable raises RuntimeError during execution in _execute_request; the exception is caught by `except RuntimeError:` and re-wrapped (the failure metric is incremented if a metrics collector is present).

Common situations: User code performing requests with requests/httpx raising RuntimeError on connection issues; generic runtime failures inside the custom Caller implementation.

Understand the failure class

Background: 'Something went wrong' / 'Request failed (500)' / 'HTTP error! status: 404' — what failed HTTP requests actually mean and how to find the real cause — this error's family across 28 libraries.

Related errors

• Timeout {timeout} exceeded while completing request: {reques
• Request to %s failed with status %d: %s
• Could not find coder for URN " + urn
• Unknown type of encoding context
• Unknown type of decoding context

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/cb896fa93dcc5542. Report an issue: GitHub.