{"record":{"id":"87edf926b61680dc","repo":"apache/beam","slug":"gcs-path-must-be-in-the-form-gs-bucket-object-encountered","errorCode":null,"errorMessage":"GCS path must be in the form gs://<bucket>/<object>. Encountered {gcs_path!r}","messagePattern":"GCS path must be in the form gs://<bucket>/<object>\\. Encountered (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/io/gcp/gcsio.py","lineNumber":69,"sourceCode":"from apache_beam.options.pipeline_options import GoogleCloudOptions\nfrom apache_beam.options.pipeline_options import PipelineOptions\n\n__all__ = ['GcsIO', 'create_storage_client']\n\n_LOGGER = logging.getLogger(__name__)\n\nDEFAULT_READ_BUFFER_SIZE = 16 * 1024 * 1024\n\n# Maximum number of operations permitted in GcsIO.copy_batch() and\n# GcsIO.delete_batch().\nMAX_BATCH_OPERATION_SIZE = 100\n\n\ndef parse_gcs_path(gcs_path, object_optional=False):\n  \"\"\"Return the bucket and object names of the given gs:// path.\"\"\"\n  match = re.match('^gs://([^/]+)/(.*)$', gcs_path)\n  if match is None or (match.group(2) == '' and not object_optional):\n    raise ValueError(\n        'GCS path must be in the form gs://<bucket>/<object>. '\n        f'Encountered {gcs_path!r}')\n  return match.group(1), match.group(2)\n\n\ndef default_gcs_bucket_name(project, region):\n  from hashlib import md5\n  return 'dataflow-staging-%s-%s' % (\n      region, md5(project.encode('utf8')).hexdigest())\n\n\ndef _get_project_number(project_id, credentials=None):\n  \"\"\"Resolves a project ID to its project number using Cloud Resource Manager API.\"\"\"\n  from google.cloud import resourcemanager_v3\n  client = resourcemanager_v3.ProjectsClient(credentials=credentials)\n  project_info = client.get_project(name=f\"projects/{project_id}\")\n  # project_info.name is of the form \"projects/PROJECT_NUMBER\"\n  return int(project_info.name.split('/')[-1])","sourceCodeStart":51,"sourceCodeEnd":87,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/io/gcp/gcsio.py#L51-L87","documentation":"parse_gcs_path raises ValueError when the given string is not a valid GCS path of the form gs://<bucket>/<object>. It requires a non-empty bucket and, unless object_optional=True, a non-empty object name.","triggerScenarios":"Passing paths like 'gs://bucket' (no object, object_optional=False), 'gs:///obj', local paths like '/tmp/file', or paths with a different scheme to parse_gcs_path or its callers (open, delete, copy, exists).","commonSituations":"Mixing local file paths with GCS paths in a pipeline; forgetting the object part of the URI; missing 'gs://' prefix after string formatting; Windows-style paths.","solutions":["Ensure the path matches gs://<bucket>/<object> with a non-empty bucket and object.","Add or fix the 'gs://' scheme prefix.","Pass object_optional=True only when bucket-only paths are intentional.","Use FullMatchGlob patterns or apache_beam.io.filesystems.FileSystems to handle mixed local/GCS paths."],"exampleFix":"// before\nparse_gcs_path('gs://my-bucket')\n// after\nparse_gcs_path('gs://my-bucket/path/to/object', object_optional=True)","handlingStrategy":"validation","validationCode":"import re\nif not re.match(r'^gs://[^/]+/.+$', path):\n    raise ValueError(f'invalid gcs path: {path!r}')","typeGuard":"def is_gcs_path(p):\n    return isinstance(p, str) and re.match(r'^gs://[^/]+/.+$', p) is not None","tryCatchPattern":"try:\n    bucket, obj = parse_gcs_path(path)\nexcept ValueError as e:\n    logging.error('bad path: %s', e)\n    return","preventionTips":["Always include both bucket and object in gs:// URIs","Use object_optional=True only when bucket-only paths are intended","Normalize scheme prefixes before calling Beam IO APIs","Keep local and GCS paths in separate code paths"],"tags":["gcs","path","validation"],"backgroundTag":"invalid-url-format","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}