{"record":{"id":"8d49de27d3bf83b7","repo":"apache/beam","slug":"scheme-must-be-s3","errorCode":null,"errorMessage":"scheme must be 's3'","messagePattern":"scheme must be 's3'","errorType":"validation","errorClass":null,"httpStatus":null,"severity":"error","filePath":"sdks/go/pkg/beam/io/filesystem/s3/util.go","lineNumber":32,"sourceCode":"// limitations under the License.\n\npackage s3\n\nimport (\n\t\"errors\"\n\t\"fmt\"\n\t\"net/url\"\n)\n\n// parseURI deconstructs the S3 uri in the format 's3://bucket/key' to (bucket, key)\nfunc parseURI(uri string) (string, string, error) {\n\tparsed, err := url.Parse(uri)\n\tif err != nil {\n\t\treturn \"\", \"\", err\n\t}\n\n\tif parsed.Scheme != \"s3\" {\n\t\treturn \"\", \"\", errors.New(\"scheme must be 's3'\")\n\t}\n\n\tbucket := parsed.Host\n\tif bucket == \"\" {\n\t\treturn \"\", \"\", errors.New(\"bucket must not be empty\")\n\t}\n\n\tvar key string\n\tif parsed.Path != \"\" {\n\t\tkey = parsed.Path[1:]\n\t}\n\n\treturn bucket, key, nil\n}\n\n// makeURI constructs an S3 uri from the bucket and key to the format 's3://bucket/key'\nfunc makeURI(bucket string, key string) string {\n\treturn fmt.Sprintf(\"s3://%s/%s\", bucket, key)","sourceCodeStart":14,"sourceCodeEnd":50,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/go/pkg/beam/io/filesystem/s3/util.go#L14-L50","documentation":"The s3 filesystem package parses URIs before any S3 operation (List, OpenRead, OpenWrite, Size, LastModified, Remove) and requires the URI scheme to be exactly 's3'. Any other scheme (http, file, gs, plain paths) is rejected because this driver only handles s3:// URLs.","triggerScenarios":"Registering/using the s3 filesystem but passing a URI like 'https://bucket/key', 'gs://bucket/key', '/local/path/file', or 'S3://bucket/key' (uppercase scheme — url.Parse lowercases, but a different scheme string like 's3a' fails) to any of the s3 filesystem functions.","commonSituations":"Hardcoding an https console URL instead of the s3 object URI; copying pipeline code between filesystem drivers (gcs/http) without changing the scheme; using s3a:// or s3n:// URIs from Hadoop-style configs; environment-driven bucket paths that default to local paths.","solutions":["Use proper s3 URIs: s3://<bucket>/<key> (e.g. s3://my-bucket/data/file.json).","Strip or translate other schemes before calling: convert https://bucket/key to s3://bucket/key.","If you need http/gcs/local access, use the corresponding filesystem package instead of the s3 driver.","Validate the URI prefix (strings.HasPrefix(uri, \"s3://\")) in your configuration code before constructing the pipeline."],"exampleFix":"// before\nuri := \"https://my-bucket.s3.amazonaws.com/data/file.json\" // wrong scheme\n\n// after\nuri := \"s3://my-bucket/data/file.json\"","handlingStrategy":"validation","validationCode":"u, err := url.Parse(uri)\nif err != nil || u.Scheme != \"s3\" || u.Host == \"\" {\n\treturn fmt.Errorf(\"expected s3://<bucket>/<key>, got %q\", uri)\n}","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Normalize all object paths to s3://bucket/key form before passing to the s3 filesystem.","Validate URI scheme and bucket in configuration-loading code.","Do not reuse https:// console URLs or gs:// / s3a:// URIs with the s3 driver."],"tags":["go","apache-beam","s3","filesystem","url-scheme"],"backgroundTag":"invalid-url-format","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}