{"record":{"id":"7c00eaf65beea691","repo":"apache/beam","slug":"artifacts-not-found-at-location-s-when-using-read-artifact","errorCode":null,"errorMessage":"Artifacts not found at location: %s when using read_artifact_location. Make sure you've run the pipeline with write_artifact_location using this artifact location before running with read_artifact_location set.","messagePattern":"Artifacts not found at location: (.+?) when using read_artifact_location\\. Make sure you've run the pipeline with write_artifact_location using this artifact location before running with read_artifact_location set\\.","errorType":"exception","errorClass":"FileNotFoundError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/ml/transforms/handlers.py","lineNumber":392,"sourceCode":"                dict[str, Union[tuple(column_type_mapping.values())]]))  # type: ignore\n        # AnalyzeAndTransformDataset raise type hint since this is\n        # schema'd PCollection and the current output type would be a\n        # custom type(NamedTuple) or a beam.Row type.\n      else:\n        column_type_mapping = self._map_column_names_to_types_from_transforms()\n        # Add id so TFT can output id as output but as a no-op.\n      raw_data_metadata = self.get_raw_data_metadata(\n          input_types=column_type_mapping)\n      # Write untransformed metadata to a file so that it can be re-used\n      # during Transform step.\n      metadata_io.write_metadata(\n          metadata=raw_data_metadata,\n          path=os.path.join(self.artifact_location, RAW_DATA_METADATA_DIR))\n    else:\n      # Read the metadata from the artifact_location.\n      if not FileSystems.exists(os.path.join(\n          self.artifact_location, RAW_DATA_METADATA_DIR, SCHEMA_FILE)):\n        raise FileNotFoundError(\n            \"Artifacts not found at location: %s when using \"\n            \"read_artifact_location. Make sure you've run the pipeline with \"\n            \"write_artifact_location using this artifact location before \"\n            \"running with read_artifact_location set.\" %\n            os.path.join(self.artifact_location, RAW_DATA_METADATA_DIR))\n      raw_data_metadata = metadata_io.read_metadata(\n          os.path.join(self.artifact_location, RAW_DATA_METADATA_DIR))\n\n      element_type = raw_data.element_type\n      if (isinstance(element_type, RowTypeConstraint) or\n          native_type_compatibility.match_is_named_tuple(element_type)):\n        # convert Row or NamedTuple to Dict\n        column_type_mapping = self._map_column_names_to_types(\n            row_type=element_type)\n        raw_data = (\n            raw_data\n            | _ConvertNamedTupleToDict().with_output_types(\n                dict[str, Union[tuple(column_type_mapping.values())]]))  # type: ignore","sourceCodeStart":374,"sourceCodeEnd":410,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/ml/transforms/handlers.py#L374-L410","documentation":"In read mode (read_artifact_location set), MLTransform's expand() must load the raw-data metadata schema written by a previous write run. If SCHEMA_FILE does not exist under RAW_DATA_METADATA_DIR inside the artifact location, a FileNotFoundError is raised because there is nothing to read from.","triggerScenarios":"Calling MLTransform(...).with_read_artifact_location(path) on a path where no prior pipeline run used with_write_artifact_location, or the path is wrong/empty, or the run that wrote artifacts failed partway.","commonSituations":"Pointing read at a GCS/local path with a typo, running read before ever running write, cleaning the artifact directory between runs, or a failed first pipeline run that never materialized the schema file.","solutions":["Run the pipeline first with with_write_artifact_location pointing at the same path so the metadata schema is produced.","Verify the exact path exists and contains <artifact_location>/RAW_DATA_METADATA_DIR/SCHEMA_FILE (use FileSystems.exists or gsutil ls).","Check for typos or permission issues in the artifact_location path.","If a previous write run failed, re-run the write pipeline to completion before reading."],"exampleFix":"# before\nresult = pcoll | MLTransform(...).with_read_artifact_location('gs://bucket/artifacts')\n\n# after\n# step 1: write\ntrain = pcoll | MLTransform(...).with_write_artifact_location('gs://bucket/artifacts')\n# step 2: read\nresult = eval_pcoll | MLTransform(...).with_read_artifact_location('gs://bucket/artifacts')","handlingStrategy":"validation","validationCode":"from apache_beam.io.filesystems import FileSystems\nimport os\nfrom apache_beam.ml.transforms.handlers import RAW_DATA_METADATA_DIR, SCHEMA_FILE\n\ndef artifacts_exist(artifact_location: str) -> bool:\n    return FileSystems.exists(os.path.join(artifact_location, RAW_DATA_METADATA_DIR, SCHEMA_FILE))\n\nassert artifacts_exist(loc), 'Run the write pipeline first'","typeGuard":"def artifacts_exist(artifact_location: str) -> bool:\n    import os\n    from apache_beam.io.filesystems import FileSystems\n    from apache_beam.ml.transforms.handlers import RAW_DATA_METADATA_DIR, SCHEMA_FILE\n    return FileSystems.exists(os.path.join(artifact_location, RAW_DATA_METADATA_DIR, SCHEMA_FILE))","tryCatchPattern":"try:\n    result = eval_pcoll | MLTransform(...).with_read_artifact_location(loc)\nexcept FileNotFoundError as e:\n    # fall back to a fresh write pass to bootstrap artifacts\n    train_pcoll | MLTransform(...).with_write_artifact_location(loc)\n    result = eval_pcoll | MLTransform(...).with_read_artifact_location(loc)","preventionTips":["Always run the write-artifact pipeline before any read-artifact pipeline.","Centralize artifact_location in config to avoid path typos.","Check FileSystems.exists on the schema file before submitting read pipelines.","Do not clean the artifact directory between write and read runs."],"tags":["python","apache-beam","artifact-location","file-not-found"],"backgroundTag":"file-not-found","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T21:17:11.552Z"}