{"record":{"id":"f9074e1ed7241bbf","repo":"apache/beam","slug":"basepath-r-must-be-s3-path","errorCode":null,"errorMessage":"Basepath %r must be S3 path.","messagePattern":"Basepath %r must be S3 path\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/io/aws/s3filesystem.py","lineNumber":65,"sourceCode":"    self._options = pipeline_options\n\n  @classmethod\n  def scheme(cls):\n    \"\"\"URI scheme for the FileSystem\n    \"\"\"\n    return 's3'\n\n  def join(self, basepath, *paths):\n    \"\"\"Join two or more pathname components for the filesystem\n\n    Args:\n      basepath: string path of the first component of the path\n      paths: path components to be added\n\n    Returns: full path after combining all of the return nulled components\n    \"\"\"\n    if not basepath.startswith(S3FileSystem.S3_PREFIX):\n      raise ValueError('Basepath %r must be S3 path.' % basepath)\n\n    path = basepath\n    for p in paths:\n      path = path.rstrip('/') + '/' + p.lstrip('/')\n    return path\n\n  def split(self, path):\n    \"\"\"Splits the given path into two parts.\n\n    Splits the path into a pair (head, tail) such that tail contains the last\n    component of the path and head contains everything up to that.\n\n    Head will include the S3 prefix ('s3://').\n\n    Args:\n      path: path as a string\n    Returns:\n      a pair of path components as strings.","sourceCodeStart":47,"sourceCodeEnd":83,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/io/aws/s3filesystem.py#L47-L83","documentation":"S3FileSystem.join() raises ValueError('Basepath %r must be S3 path.') when the first path component does not start with the S3 prefix (s3://). The method only knows how to build S3 URLs, so a basepath like a local path or bare bucket name is rejected before joining.","triggerScenarios":"Calling S3FileSystem().join(basepath, *paths) where basepath lacks the 's3://' prefix — e.g. passing '/tmp/data', 'bucket/key', or a GCS path.","commonSituations":"Mixing local and S3 paths in file staging code; generic filesystem helpers passing any path into a hardcoded S3FileSystem; typos like 's3:/bucket' or stripped prefixes; tests that forget to prefix fixture paths.","solutions":["Ensure the basepath starts with S3FileSystem.S3_PREFIX ('s3://') before calling join.","Prepend the prefix: S3FileSystem.S3_PREFIX + path if you know the path is an S3 reference.","Normalize with match_basepath/strip_prefix helpers or validate the URL with FileSystemURI helpers first.","Use a generic filesystem (FileSystems.get_scheme-based) so join dispatches to the right filesystem."],"exampleFix":"// before\nS3FileSystem().join('bucket/output', 'part-0.avro')\n// after\nS3FileSystem().join('s3://bucket/output', 'part-0.avro')","handlingStrategy":"validation","validationCode":"def ensure_s3_path(basepath: str) -> str:\n    return basepath if basepath.startswith('s3://') else 's3://' + basepath.lstrip('/')","typeGuard":"def is_s3_path(p: str) -> bool:\n    return isinstance(p, str) and p.startswith('s3://')","tryCatchPattern":"try:\n    full = S3FileSystem().join(basepath, *parts)\nexcept ValueError as e:\n    if 'must be S3 path' in str(e):\n        full = S3FileSystem().join('s3://' + basepath.lstrip('/'), *parts)\n    else:\n        raise","preventionTips":["Always store S3 targets as full s3:// URIs in config","Avoid hand-stripping scheme prefixes (lstrip('s3:/'))","Route by scheme with FileSystems instead of hardcoding S3FileSystem"],"tags":["s3","path","aws","io","python"],"backgroundTag":"invalid-url-format","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}