apache/beam · error · ValueError

Invalid path

Error message

Invalid path: %s

What it means

After verifying the 'gs://' prefix, GCSFileSystem.split locates the last '/' after the prefix to separate directory from file name. If the separator sits exactly at the prefix boundary (last_sep == 0, i.e. 'gs://filename' with no bucket/key structure), the path is structurally invalid for GCS and the library raises this ValueError.

Solutions

1. Pass a fully formed path 'gs://bucket/path/to/object' with at least a bucket and object name.
2. Validate that path is longer than len('gs://') and contains a '/' after 'gs://' before calling split.
3. Trace where the path was truncated (often an empty/missing bucket in config).

Example fix

# before
GCSFileSystem.split('gs://mykey')
# after
GCSFileSystem.split('gs://my-bucket/mykey')

def assert_splittable_gcs_path(path):
    prefix = 'gs://'
    p = path.strip()
    if not p.startswith(prefix) or p.rfind('/') <= len(prefix) - 1:
        raise ValueError('Path must be gs://bucket/key form: %r' % path)
    return p

def is_splittable_gcs_path(path):
    return isinstance(path, str) and path.strip().startswith('gs://') and '/' in path.strip()[5:]

try:
    d, n = fs.split(path)
except ValueError as e:
    logging.error('Malformed GCS path (missing bucket/key): %s', e)
    raise

When it happens

Trigger: Calling GCSFileSystem.split('gs://name') where nothing follows the 'gs://' slash; a path like 'gs://' or a malformed 'gs:/x' that passed the prefix check but has no bucket/key separator.

Common situations: Truncated or empty GCS path variables from misconfigured pipeline options; string slicing bugs upstream that stripped the bucket from 'gs://bucket/key'.

Understand the failure class

Background: "Invalid ... format", "must be in format X", "does not look like a ..." — invalid argument format errors across CLI tools and libraries — this error's family across 17 libraries.

Related errors

• Basepath %r must be GCS path.
• Destination %r must be GCS path.
• Path %r must be GCS path.
• cache_root GCS bucket path is invalid.
• combine_fn must be specified.

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/0c86183e44b3cd19. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/io/gcp/gcsfilesystem.py:102

      path: path as a string
    Returns:
      a pair of path components as strings.
    """
    path = path.strip()
    if not path.startswith(GCSFileSystem.GCS_PREFIX):
      raise ValueError('Path %r must be GCS path.' % path)

    prefix_len = len(GCSFileSystem.GCS_PREFIX)
    last_sep = path[prefix_len:].rfind('/')
    if last_sep >= 0:
      last_sep += prefix_len

    if last_sep > 0:
      return (path[:last_sep], path[last_sep + 1:])
    elif last_sep < 0:
      return (path, '')
    else:
      raise ValueError('Invalid path: %s' % path)

  def mkdirs(self, path):
    """Recursively create directories for the provided path.

    Args:
      path: string path of the directory structure that should be created

    Raises:
      IOError: if leaf directory already exists.
    """
    pass

  def has_dirs(self):
    """Whether this FileSystem supports directories."""
    return False

  def _list(self, dir_or_prefix):
    """List files in a location.

View on GitHub (pinned to 12126d8942)