{"record":{"id":"560a073e27499df0","repo":"apache/beam","slug":"source-blob-s-not-found-during-copying","errorCode":null,"errorMessage":"source blob %s not found during copying","messagePattern":"source blob (.+?) not found during copying","errorType":"exception","errorClass":"NotFound","httpStatus":404,"severity":"error","filePath":"sdks/python/apache_beam/io/gcp/gcsio.py","lineNumber":450,"sourceCode":"    return final_results\n\n  def copy(self, src, dest):\n    \"\"\"Copies the given GCS object from src to dest.\n\n    Args:\n      src: GCS file path pattern in the form gs://<bucket>/<name>.\n      dest: GCS file path pattern in the form gs://<bucket>/<name>.\n\n    Raises:\n      Any exceptions during copying\n    \"\"\"\n    src_bucket_name, src_blob_name = parse_gcs_path(src)\n    dest_bucket_name, dest_blob_name= parse_gcs_path(dest, object_optional=True)\n    src_bucket = self.client.bucket(src_bucket_name)\n    if self._use_blob_generation:\n      src_blob = src_bucket.get_blob(src_blob_name)\n      if src_blob is None:\n        raise NotFound(\"source blob %s not found during copying\" % src)\n      src_generation = src_blob.generation\n    else:\n      src_blob = src_bucket.blob(src_blob_name)\n      src_generation = None\n    dest_bucket = self.client.bucket(dest_bucket_name)\n    if not dest_blob_name:\n      dest_blob_name = None\n    src_bucket.copy_blob(\n        src_blob,\n        dest_bucket,\n        new_name=dest_blob_name,\n        source_generation=src_generation,\n        retry=self._storage_client_retry)\n\n  def _copy_batch_request(self, pair):\n    src_bucket_name, src_blob_name = parse_gcs_path(pair[0])\n    dest_bucket_name, dest_blob_name = parse_gcs_path(pair[1])\n    src_bucket = self.client.bucket(src_bucket_name)","sourceCodeStart":432,"sourceCodeEnd":468,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/io/gcp/gcsio.py#L432-L468","documentation":"GcsIO.copy raises google.cloud.exceptions.NotFound ('source blob %s not found during copying') when _use_blob_generation is enabled and the source blob cannot be fetched before copying. Beam reads the source generation to make the copy generation-safe; a missing source aborts the copy.","triggerScenarios":"Calling GcsIO.copy (directly or via copytree/rename) with a source gs:// path whose object does not exist while blob-generation mode is enabled.","commonSituations":"Renaming files already moved/deleted by a concurrent job; typo'd source object names; copying objects deleted by lifecycle rules before the copy runs.","solutions":["Verify the source object exists (gcsio.exists / blob listing) before copying.","Correct the source gs:// path (bucket and object name).","Catch NotFound and skip/recreate the missing source as appropriate.","If generation safety is not required, disable _use_blob_generation so a stale-generation reference isn't needed (still fails on missing object at copy time)."],"exampleFix":"// before\ngcsio.copy(src, dst)\n// after\nif gcsio.exists(src):\n  gcsio.copy(src, dst)\nelse:\n  logging.warning('skip missing %s', src)","handlingStrategy":"validation","validationCode":"if not gcsio.exists(src):\n    raise FileNotFoundError(src)","typeGuard":"def is_gcs_path(p):\n    return isinstance(p, str) and re.match(r'^gs://[^/]+/.+$', p) is not None","tryCatchPattern":"from google.cloud.exceptions import NotFound\ntry:\n    gcsio.copy(src, dst)\nexcept NotFound:\n    logging.warning('source missing, skipping: %s', src)","preventionTips":["Check source existence before copy/rename","Watch for concurrent jobs deleting sources","Account for GCS lifecycle rules","Validate object names (exact casing)"],"tags":["gcs","copy","not-found"],"backgroundTag":"resource-not-found","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}