{"record":{"id":"a41d5ddeaf5621f1","repo":"apache/beam","slug":"pcollection-not-available-please-run-the-pipeline","errorCode":null,"errorMessage":"PCollection not available, please run the pipeline.","messagePattern":"PCollection not available, please run the pipeline\\.","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/runners/interactive/interactive_runner.py","lineNumber":353,"sourceCode":"    WindowedValues. Otherwise, return the element as itself.\n    \"\"\"\n    return list(self.read(pcoll, include_window_info))\n\n  def read(self, pcoll, include_window_info=False):\n    \"\"\"Reads the PCollection one element at a time from cache.\n\n    If include_window_info is True, then returns the elements as\n    WindowedValues. Otherwise, return the element as itself.\n    \"\"\"\n    key = self._pipeline_instrument.cache_key(pcoll)\n    cache_manager = ie.current_env().get_cache_manager(\n        self._pipeline_instrument.user_pipeline)\n    if key and cache_manager.exists('full', key):\n      coder = cache_manager.load_pcoder('full', key)\n      reader, _ = cache_manager.read('full', key)\n      return to_element_list(reader, coder, include_window_info)\n    else:\n      raise ValueError('PCollection not available, please run the pipeline.')\n\n  def cancel(self):\n    self._underlying_result.cancel()\n","sourceCodeStart":335,"sourceCodeEnd":357,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/runners/interactive/interactive_runner.py#L335-L357","documentation":"CacheManager-backed PCollection readers (used by ib.collect / recording get()) materialize elements from a 'full' cache written when the pipeline ran. read() checks cache_manager.exists('full', key); if the cached data for this PCollection's key is missing it raises this ValueError instead of silently returning empty data. It means the pipeline containing this PCollection has not been executed (or its cache was cleared) in the current interactive session.","triggerScenarios":"Calling ib.collect(pcoll) (which routes through RecordingManager.get → read) before running the pipeline that produces pcoll; the cache directory was deleted or cache_root changed between runs; the pipeline ran but this PCollection was outside the recorded/instrumented set; kernel restart lost the recording while cache files were cleaned.","commonSituations":"Notebooks: editing a pcoll definition after the last run then collecting without re-running; cache_root pointing to ephemeral storage cleared between sessions; using ib.collect on a pcoll from a pipeline built with a non-interactive runner so nothing was recorded; cache cleanup removed files.","solutions":["Run (or re-run) the pipeline with the InteractiveRunner so the PCollection is recorded into the cache, then call get/collect again.","Re-execute ib.show/ib.compute on the pcoll to create a fresh recording, then collect.","Check your cache_root directory for the pcoll's cache files to confirm the recording exists.","Don't change ib.options.cache_root between defining and collecting a pcoll.","Re-run all upstream cells after a kernel restart so watches and recordings are re-established."],"exampleFix":"// before: pc = p | beam.Map(lambda x: x); ib.collect(pc)  # pipeline never ran | // after: pc = p | beam.Map(lambda x: x); ib.show(pc)  # runs/records; then ib.collect(pc)","handlingStrategy":"fallback","validationCode":"def has_recording(pcoll): return ib.current_env().get_recording_manager(pcoll.pipeline, create_if_absent=False) is not None; assert has_recording(pc)  # else run ib.show/ib.compute first","typeGuard":"def is_recorded(pcoll): return ib.current_env().get_recording_manager(pcoll.pipeline, create_if_absent=False) is not None","tryCatchPattern":"try: rows = ib.collect(pc) | except ValueError as e: (ib.compute(pc); rows = ib.collect(pc)) if 'PCollection not available' in str(e) else raise","preventionTips":["Always run/record (ib.show or ib.compute) a pcoll before ib.collect on it.","Don't change ib.options.cache_root between recording and reading.","Re-run pipeline cells after kernel restarts.","Keep the cache directory intact between the run and the read."],"tags":["python","apache-beam","interactive","cache","state","value-error"],"backgroundTag":"resource-not-found","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T11:17:12.474Z"}