apache/beam · error · ValueError

Basepath %r must be GCS path.

Error message

Basepath %r must be GCS path.

What it means

GCSFileSystem.join is the GCS FileSystem implementation of path joining and requires the base path to start with 'gs://'. If a non-GCS base path (local path, s3://, plain relative path) is passed as basepath, it raises this ValueError.

Solutions

1. Ensure the basepath starts with 'gs://' before calling join (e.g. basepath = 'gs://' + path if missing).
2. Use FileSystem.join via FileSystems.get_filesystem(path) so local paths route to the LocalFileSystem instead.
3. Validate the temp/staging location in pipeline options is a full gs:// URL.

Example fix

# before
GCSFileSystem.join(temp_location, 'checksums.json')  # temp_location='/tmp/beam'
# after
GCSFileSystem.join('gs://my-bucket/staging', 'checksums.json')

def assert_gcs_basepath(basepath):
    if not basepath.startswith('gs://'):
        raise ValueError('Basepath must be gs:// path: %r' % basepath)
    return basepath

def is_gcs_path(path):
    return isinstance(path, str) and path.startswith('gs://')

try:
    full = fs.join(basepath, name)
except ValueError as e:
    logging.error('Non-GCS basepath for GCS join: %s', e)
    raise

When it happens

Trigger: Calling gcsfilesystem.GCSFileSystem.join('/local/dir', 'file.txt') or join('s3://bucket/x', 'y'); or passing a user-supplied output path that was not normalized to gs:// before match/join logic.

Common situations: Using Beam FileSystems / match APIs with local paths but resolving them through the GCS file system implementation; config where staging_location or temp_location lost its gs:// prefix.

Understand the failure class

Background: "Invalid URL" errors: why new URL(), URI.parse, and reqwest::Url reject your string — missing scheme, whitespace, and bad path format — this error's family across 39 libraries.

Related errors

• Destination %r must be GCS path.
• Invalid path
• Path %r must be GCS path.
• cache_root GCS bucket path is invalid.
• combine_fn must be specified.

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/e17a76d043388e89. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/io/gcp/gcsfilesystem.py:69

    self._pipeline_options = pipeline_options

  @classmethod
  def scheme(cls):
    """URI scheme for the FileSystem
    """
    return 'gs'

  def join(self, basepath, *paths):
    """Join two or more pathname components for the filesystem

    Args:
      basepath: string path of the first component of the path
      paths: path components to be added

    Returns: full path after combining all the passed components
    """
    if not basepath.startswith(GCSFileSystem.GCS_PREFIX):
      raise ValueError('Basepath %r must be GCS path.' % basepath)
    path = basepath
    for p in paths:
      path = path.rstrip('/') + '/' + p.lstrip('/')
    return path

  def split(self, path):
    """Splits the given path into two parts.

    Splits the path into a pair (head, tail) such that tail contains the last
    component of the path and head contains everything up to that.

    Head will include the GCS prefix ('gs://').

    Args:
      path: path as a string
    Returns:
      a pair of path components as strings.
    """

View on GitHub (pinned to 12126d8942)