{"record":{"id":"0b2c544b42f16407","repo":"apache/beam","slug":"materialized-pipeline-is-not-allocated-for-result-cache","errorCode":null,"errorMessage":"Materialized pipeline is not allocated for result cache.","messagePattern":"Materialized pipeline is not allocated for result cache\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/transforms/ptransform.py","lineNumber":166,"sourceCode":"}  # type: dict[tuple[int, int], dict[int, _MaterializedResult]]\n_pipeline_materialization_lock = threading.Lock()\n\n\ndef _allocate_materialized_pipeline(pipeline):\n  # type: (Pipeline) -> None\n  pid = os.getpid()\n  with _pipeline_materialization_lock:\n    pipeline_id = id(pipeline)\n    _pipeline_materialization_cache[(pid, pipeline_id)] = {}\n\n\ndef _allocate_materialized_result(pipeline):\n  # type: (Pipeline) -> _MaterializedResult\n  pid = os.getpid()\n  with _pipeline_materialization_lock:\n    pipeline_id = id(pipeline)\n    if (pid, pipeline_id) not in _pipeline_materialization_cache:\n      raise ValueError(\n          'Materialized pipeline is not allocated for result '\n          'cache.')\n    result_id = len(_pipeline_materialization_cache[(pid, pipeline_id)])\n    result = _MaterializedResult(pipeline_id, result_id)\n    _pipeline_materialization_cache[(pid, pipeline_id)][result_id] = result\n    return result\n\n\ndef _get_materialized_result(pipeline_id, result_id):\n  # type: (int, int) -> _MaterializedResult\n  pid = os.getpid()\n  with _pipeline_materialization_lock:\n    if (pid, pipeline_id) not in _pipeline_materialization_cache:\n      raise Exception(\n          'Materialization in out-of-process and remote runners is not yet '\n          'supported.')\n    return _pipeline_materialization_cache[(pid, pipeline_id)][result_id]\n","sourceCodeStart":148,"sourceCodeEnd":184,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/transforms/ptransform.py#L148-L184","documentation":"During pipeline materialization, a pipeline must first be registered in the in-process _pipeline_materialization_cache by _materialize_transform before results can be allocated. If _allocate_materialized_result is called with a pipeline that was never registered for the current process, this internal invariant ValueError fires.","triggerScenarios":"Calling a materializing API (e.g. beam.Materialize / deferred materialization) with a pipeline object that never went through _materialize_transform in the same process, or after the cache was cleared.","commonSituations":"Using materialization across process boundaries (fork/spawn), mixing pipelines constructed in different processes, internal Beam bugs.","solutions":["Use the materialization API only through its supported entry points (e.g. beam.Materialize on a running pipeline).","Ensure the pipeline is created and materialized in the same OS process.","Report a bug if it occurs in supported runner usage; this is an internal invariant."],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"try:\n  result = beam.Materialize(pipeline, ...)\nexcept ValueError as e:\n  if 'not allocated for result cache' in str(e):\n    raise RuntimeError('Pipeline must be materialized in the same process; recreate and materialize in one process') from e\n  raise","preventionTips":["Materialize pipelines in the same process that created them","Avoid forking between pipeline creation and materialization"],"tags":["python","apache-beam","internal"],"backgroundTag":"internal-invariant-violation","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}