apache/beam · error · RuntimeError

Full trace: , \n output of the failed child process

Error message

Full trace: {}, \n output of the failed child process {} 

What it means

Generic branch of check_output for non-pip subprocess failures: when any other command exits non-zero, Beam re-raises as RuntimeError with the full traceback and the child process's output. It exists so failed external commands surface their stderr/stdout instead of being swallowed.

Solutions

1. Inspect 'output of the failed child process' in the message for the real error
2. Run the same command manually on a worker to reproduce
3. Confirm the executable exists on worker PATH (or the OSError branch would have fired)
4. Fix the command's arguments/exit condition, or handle the non-zero exit explicitly

Example fix

// before
out = processes.check_output(('gsutil', 'cat', path))
// after
try:
    out = processes.check_output(('gsutil', 'cat', path))
except RuntimeError as e:
    logging.warning('gsutil failed: %s', e)
    out = b''

import shutil
assert shutil.which(cmd[0]), f'{cmd[0]} not on PATH'

try:
    out = processes.check_output(cmd)
except RuntimeError as e:
    logging.error('command failed: %s', e)
    # fallback or re-raise

When it happens

Trigger: Calling processes.check_output with any command tuple/args that is not pip and exits with a non-zero return code (CalledProcessError).

Common situations: Shelling out to git, gsutil, or custom binaries from pipeline code; commands missing on worker PATH (falls into the else branch when args shape differs); scripts returning non-zero exit codes.

Related errors

• Aborted with error
• Could not generate external transform wrappers due to error
• Could not generate protos due to error
• Executable not found
• Full trace: \n Output of the failed child process

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/07d2be5139d390c6. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/utils/processes.py:99

          \n Output of the failed child process: {}" \
          .format(traceback.format_exc(), error.output)) from error
    return out

  def check_output(*args, **kwargs):
    if force_shell:
      kwargs['shell'] = True
    try:
      out = subprocess.check_output(*args, **kwargs)
    except OSError as e:
      raise RuntimeError("Executable {} not found".format(args[0])) from e
    except subprocess.CalledProcessError as error:
      if isinstance(args, tuple) and (args[0][2] == "pip"):
        raise RuntimeError( \
          "Full traceback: {} \n Pip install failed for package: {} \
          \n Output from execution of subprocess: {}" \
          .format(traceback.format_exc(), args[0][6], error.output)) from error
      else:
        raise RuntimeError("Full trace: {}, \
           output of the failed child process {} "\
          .format(traceback.format_exc(), error.output)) from error
    return out

  def Popen(*args, **kwargs):
    if force_shell:
      kwargs['shell'] = True
    return subprocess.Popen(*args, **kwargs)

View on GitHub (pinned to 12126d8942)