apache/beam · error · RuntimeError

Cannot record because a dependency failed to compute…

Error message

Cannot record because a dependency failed to compute asynchronously.

What it means

Recording.record requires all dependency PCollections to be computed before recording starts; if _wait_for_dependencies returns False after waiting on uncomputed inputs, it raises this RuntimeError rather than recording from a pipeline whose inputs never materialized.

Solutions

1. Compute the upstream PCollections successfully (ib.collect) before starting the recording
2. Inspect why the dependency computation failed (runner logs) and fix the root cause
3. Restart the interactive environment to clear stale failed-computation state
4. Record only PCollections whose full lineage has already been computed

Example fix

// before
recording = ib.recordings.record([downstream_pcoll], max_n=100)
// after
ib.collect(upstream_pcoll)  # ensure dependencies are computed
recording = ib.recordings.record([downstream_pcoll], max_n=100)

env = ie.current_env()
uncomputed = [p for p in pcolls if not env.is_pcollection_computed(p)]
if uncomputed:
    for p in upstream_of(uncomputed):
        ib.collect(p)  # compute dependencies before recording

When it happens

Trigger: Calling ib.recordings.record (or watch) with PCollections that depend on uncomputed upstreams whose async computation failed or was cancelled.

Common situations: Notebooks setting up recordings on pipelines whose upstream ib.collect previously failed; recordings created immediately after an interrupted computation.

Understand the failure class

Background: "Invalid state transition" errors: "status must be X, actually Y", "already rejected/charging/uninstalled", "cannot ... while running" — what they mean when a library rejects your call — this error's family across 31 libraries.

Related errors

• Dependency computation failed or was cancelled.
• Asynchronous computation was cancelled.
• Blocking computation failed. State
• Dependencies must be a list of strings, got
• error installing dependencies

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/fb59452fc34649a4. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/runners/interactive/recording_manager.py:918

    # Make sure that all PCollections to be shown are watched. If a PCollection
    # has not been watched, make up a variable name for that PCollection and
    # watch it. No validation is needed here because the watch logic can handle
    # arbitrary variables.
    self._watch(pcolls)
    self.record_pipeline()

    # Early check and return if everything is already computed
    uncomputed_pcolls = self._get_uncomputed_pcolls(pcolls)
    if not uncomputed_pcolls:
      recording = Recording(
          self.user_pipeline, pcolls, None, max_n, max_duration_secs)
      self._recordings.add(recording)
      return recording

    # Wait for dependencies if there are uncomputed PCollections
    if not self._wait_for_dependencies(uncomputed_pcolls):
      raise RuntimeError(
          'Cannot record because a dependency failed to compute'
          ' asynchronously.')

    # Re-evaluate uncomputed PCollections
    uncomputed_pcolls = self._get_uncomputed_pcolls(pcolls)
    if not uncomputed_pcolls:
      recording = Recording(
          self.user_pipeline, pcolls, None, max_n, max_duration_secs)
      self._recordings.add(recording)
      return recording

    # Flattened execution path (no indentation needed)
    self._clear()

    merged_options = pipeline_options.PipelineOptions(
        **{
            **self.user_pipeline.options.get_all_options(
                drop_default=True, retain_unknown_options=True),

View on GitHub (pinned to 12126d8942)