apache/beam · error

error parsing S3 destination uri

Error message

error parsing S3 destination uri %s: %v

What it means

Returned by fs.Copy when the DESTINATION path (newpath) cannot be parsed as a valid S3 URI. parseURI requires an s3://bucket/key form; invalid schemes or missing components are rejected before CopyObject is invoked.

Solutions

1. Check that newpath is a well-formed s3://bucket/key URI.
2. Fix destination-name construction so bucket and key are always present.
3. Validate both source and destination URIs before calling Copy.

Example fix

// before
dest := bucket + "/" + key // no scheme
fsys.Copy(ctx, src, dest)

// after
dest := "s3://" + bucket + "/" + key
fsys.Copy(ctx, src, dest)

func validS3URI(p string) error {
    if !strings.HasPrefix(p, "s3://") { return fmt.Errorf("missing s3:// scheme: %q", p) }
    rest := strings.TrimPrefix(p, "s3://")
    bucket, key, ok := strings.Cut(rest, "/")
    if !ok || bucket == "" || key == "" { return fmt.Errorf("need s3://bucket/key, got %q", p) }
    return nil
}

if err := validS3URI(newpath); err != nil { return fmt.Errorf("copy destination invalid: %w", err) }
if err := fsys.Copy(ctx, oldpath, newpath); err != nil { return err }

When it happens

Trigger: Calling fs.Copy with a newpath that is not a valid S3 URI (wrong scheme, missing bucket, or empty key).

Common situations: Constructing destination names by string concatenation that omit the s3:// prefix; empty destination variables; Windows-style paths used as destinations.

Understand the failure class

Background: "Invalid URL" errors: why new URL(), URI.parse, and reqwest::Url reject your string — missing scheme, whitespace, and bad path format — this error's family across 39 libraries.

Related errors

• error parsing S3 source uri
• error parsing S3 uri
• error parsing S3 uri
• Param validation failed on part number
• All parts but the last must be larger than

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/41420ac8ee41d30f. Report an issue: GitHub.

Appendix: source

Thrown at sdks/go/pkg/beam/io/filesystem/s3/s3.go:218

	}
	if _, err = f.client.DeleteObject(ctx, params); err != nil {
		return fmt.Errorf("error deleting object %s: %v", filename, err)
	}

	return nil
}

// Copy copies the file from the old path to the new path.
func (f *fs) Copy(ctx context.Context, oldpath, newpath string) error {
	sourceBucket, sourceKey, err := parseURI(oldpath)
	if err != nil {
		return fmt.Errorf("error parsing S3 source uri %s: %v", oldpath, err)
	}

	copySource := fmt.Sprintf("%s/%s", sourceBucket, sourceKey)
	destBucket, destKey, err := parseURI(newpath)
	if err != nil {
		return fmt.Errorf("error parsing S3 destination uri %s: %v", newpath, err)
	}

	params := &s3.CopyObjectInput{
		Bucket:     aws.String(destBucket),
		CopySource: aws.String(copySource),
		Key:        aws.String(destKey),
	}
	if _, err = f.client.CopyObject(ctx, params); err != nil {
		return fmt.Errorf("error copying object %s: %v", oldpath, err)
	}

	return nil
}

// Compile time check for interface implementations.
var (
	_ filesystem.LastModifiedGetter = (*fs)(nil)
	_ filesystem.Remover            = (*fs)(nil)

View on GitHub (pinned to 12126d8942)