{"record":{"id":"88e89b7f6729d43b","repo":"apache/beam","slug":"could-not-upload-to-gcs-path-s-s-please-verify-that-sdk","errorCode":null,"errorMessage":"Could not upload to GCS path %s: %s. Please verify that credentials are valid and that you have write access to the specified path.","messagePattern":"Could not upload to GCS path (.+?): (.+?)\\. Please verify that credentials are valid and that you have write access to the specified path\\.","errorType":"exception","errorClass":"IOError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/runners/portability/sdk_container_builder.py","lineNumber":320,"sourceCode":"        (time.time() - now))\n    _LOGGER.info(\n        \"Python SDK container built and pushed as %s.\" % container_image_name)\n\n  def _upload_to_gcs(self, local_file_path, gcs_location):\n    bucket_name, blob_name = self._get_gcs_bucket_and_name(gcs_location)\n    _LOGGER.info('Starting GCS upload to %s...', gcs_location)\n    from google.cloud import storage\n    from google.cloud.exceptions import Forbidden\n    from google.cloud.exceptions import NotFound\n    try:\n      bucket = self._storage_client.get_bucket(bucket_name)\n      blob = bucket.get_blob(blob_name)\n      if not blob:\n        blob = storage.Blob(name=blob_name, bucket=bucket)\n      blob.upload_from_filename(local_file_path)\n    except Exception as e:\n      if isinstance(e, (Forbidden, NotFound)):\n        raise IOError((\n            'Could not upload to GCS path %s: %s. Please verify '\n            'that credentials are valid and that you have write '\n            'access to the specified path.') % (gcs_location, e.message))\n      raise\n    _LOGGER.info('Completed GCS upload to %s.', gcs_location)\n\n  def _get_cloud_build_id_and_log_url(self, metadata):\n    # google-cloud-build 3.35+\n    if getattr(metadata, 'build', None):\n      build = metadata.build\n      return (build.id, build.log_url)\n    # Fallback for older clients that use additionalProperties.\n    id = None\n    log_url = None\n    additional_props = getattr(metadata, 'additionalProperties', None)\n    if additional_props:\n      for item in additional_props:\n        if item.key == 'build':","sourceCodeStart":302,"sourceCodeEnd":338,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/runners/portability/sdk_container_builder.py#L302-L338","documentation":"_upload_to_gcs uploads a local file to a GCS bucket via the google-cloud-storage client before a Cloud Build. If the upload raises Forbidden or NotFound, the library converts it to IOError with guidance to check credentials and write access; any other exception is re-raised unchanged.","triggerScenarios":"_invoke_docker_build_and_push calls _upload_to_gcs with a gcs_location whose bucket doesn't exist (NotFound) or whose credentials lack storage.objects.create on it (Forbidden) during blob.upload_from_filename.","commonSituations":"Application Default Credentials not set up (gcloud auth application-default login); service account lacking Storage Object Creator role; bucket name typo or bucket in another project; staging location path misconfigured in pipeline options.","solutions":["Verify the GCS path and bucket name exist (gsutil ls <path> or gcloud storage ls)","Set up valid credentials: gcloud auth application-default login, or a service account key with Storage write access","Grant roles/storage.objectCreator on the bucket to the identity in use","Confirm storage.googleapis.com API access/network if errors persist"],"exampleFix":"// before\nbuilder = SdkContainerImageBuilder(..., gcs_location='gs://wrong-bucket/sdk.tar')\n// after\ngcloud auth application-default login\ngsutil mb gs://my-staging-bucket\nbuilder = SdkContainerImageBuilder(..., gcs_location='gs://my-staging-bucket/sdk.tar')","handlingStrategy":"validation","validationCode":"from google.cloud import storage\nfrom google.api_core.exceptions import Forbidden, NotFound\nclient = storage.Client()\ntry:\n    bucket = client.get_bucket('my-staging-bucket')\n    bucket.test_iam_permissions(['storage.objects.create'])\nexcept (Forbidden, NotFound) as e:\n    raise SystemExit(f'GCS staging path unusable before pipeline start: {e}')","typeGuard":"def gcs_path_writable(client, gcs_location: str) -> bool:\n    from urllib.parse import urlparse\n    p = urlparse(gcs_location)\n    if p.scheme != 'gs' or not p.netloc:\n        return False\n    try:\n        b = client.get_bucket(p.netloc)\n        return bool(b.test_iam_permissions(['storage.objects.create']))\n    except Exception:\n        return False","tryCatchPattern":"try:\n    builder.build_container_image(...)\nexcept IOError as e:\n    if 'Could not upload to GCS path' in str(e):\n        logger.error('Check credentials/write access: %s', e)\n        raise SystemExit(7)\n    raise","preventionTips":["Run gcloud auth application-default login before local runs","Grant roles/storage.objectCreator on the staging bucket to the active identity","Validate bucket existence with gsutil ls before launching pipelines","Keep staging location and project consistent in pipeline options"],"tags":["apache-beam","gcs","upload","permissions","google-cloud-storage"],"backgroundTag":"permission-denied","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}