What causes "unable to parse file %v" in golang-migrate/migrate?

List objects under the S3 prefix and find the two keys sharing the same parsed version; delete or re-version one.

unable to parse file %v

Error message

unable to parse file %v

What it means

While listing S3 objects, loadMigrations parses each filename with source.DefaultParse. Files that don't look like migrations are silently skipped, but if a filename DOES parse and Append still fails — meaning another migration with the same version was already added (duplicate version) — the driver returns this error wrapping the full object key. Append only fails on duplicate versions, not parse errors.

Source

Thrown at source/aws_s3/s3.go:94

}

func (s *s3Driver) loadMigrations() error {
	output, err := s.s3client.ListObjects(&s3.ListObjectsInput{
		Bucket:    aws.String(s.config.Bucket),
		Prefix:    aws.String(s.config.Prefix),
		Delimiter: aws.String("/"),
	})
	if err != nil {
		return err
	}
	for _, object := range output.Contents {
		_, fileName := path.Split(aws.StringValue(object.Key))
		m, err := source.DefaultParse(fileName)
		if err != nil {
			continue
		}
		if !s.migrations.Append(m) {
			return fmt.Errorf("unable to parse file %v", aws.StringValue(object.Key))
		}
	}
	return nil
}

func (s *s3Driver) Close() error {
	return nil
}

func (s *s3Driver) First() (uint, error) {
	v, ok := s.migrations.First()
	if !ok {
		return 0, os.ErrNotExist
	}
	return v, nil
}

func (s *s3Driver) Prev(version uint) (uint, error) {

View on GitHub (pinned to 01a9643f14)

Solutions

List objects under the S3 prefix and find the two keys sharing the same parsed version; delete or re-version one.
Narrow the S3 URL prefix (e.g. s3://bucket/migrations/current) so nested/legacy directories are not included in the listing.
Ensure version prefixes in filenames are unique across the bucket path, including zero-padding differences (1_ vs 001_).
Re-upload cleaned migration files and retry the migration.

Example fix

// before
migrations/1_add_users.up.sql
migrations/archive/1_add_users.up.sql  // duplicate version under same prefix
// after
migrations/1_add_users.up.sql
migrations/archive/2023_add_users.up.sql

Defensive patterns

Strategy: validation

Validate before calling

// dedupe by parsed version before WithInstance
seen := map[uint]bool{}
for _, key := range s3Keys {
    _, fname := path.Split(key)
    m, err := source.DefaultParse(fname)
    if err != nil || m == nil {
        continue
    }
    v := m.Version
    if seen[v] {
        return fmt.Errorf("duplicate version %d in s3 prefix: %s and earlier file", v, key)
    }
    seen[v] = true
}

Try / catch

d, err := s3Source.WithInstance(ctx)
if err != nil && strings.HasPrefix(err.Error(), "unable to parse file") {
    key := strings.TrimPrefix(err.Error(), "unable to parse file ")
    // inspect/remove the duplicate object named by key, then retry
    return fmt.Errorf("duplicate migration version for object %s", key)
}

Prevention

Use a precise S3 prefix that contains only the active migrations directory.
Keep zero-padded, unique version prefixes across the whole prefix (incl. nested folders).
Delete archived/duplicated migrations from the listed prefix.
Lint bucket contents in CI for duplicate parsed versions.

When it happens

Trigger: s3:// URL used with WithInstance where the bucket/prefix contains two objects whose basenames resolve to the same migration version, e.g. `1_foo.up.sql` in two folders within the prefix, or `1_foo.up.sql` and `001_bar.up.sql`.

Common situations: Bucket prefix overlapping nested directories (prefix `migrations/` also matching `migrations-old/` listing); copied/duplicated migration files; case-sensitivity mishaps creating near-duplicates; an old migration re-uploaded under a different name but same version.

Understand the failure class

Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

Related errors

AI-assisted analysis of golang-migrate/migrate@01a9643f14 (2026-09-02). Data as JSON: /api/errors/af4d8e726fceeebe. Report an issue: GitHub.