{"record":{"id":"204835702e67e1ae","repo":"ipfs/kubo","slug":"external-migration-phase-failed-w","errorCode":null,"errorMessage":"external migration phase failed: %w","messagePattern":"external migration phase failed: %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"critical","filePath":"repo/fsrepo/migrations/migrations.go","lineNumber":435,"sourceCode":"\t\tlogger.Printf(\"Phase 1: External migration from v%d to v%d\", currentVer, embeddedMigrationsMinVersion)\n\n\t\t// Check for external migration binaries in PATH first\n\t\tmigrations, binPaths, err := findMigrations(ctx, currentVer, embeddedMigrationsMinVersion)\n\t\tif err != nil {\n\t\t\treturn fmt.Errorf(\"could not determine external migration paths: %w\", err)\n\t\t}\n\n\t\tfoundAll := true\n\t\tfor _, migName := range migrations {\n\t\t\tif _, exists := binPaths[migName]; !exists {\n\t\t\t\tfoundAll = false\n\t\t\t\tbreak\n\t\t\t}\n\t\t}\n\n\t\tif foundAll {\n\t\t\tif err = runMigrationsFromPath(ctx, migrations, binPaths, ipfsDir, logger, false); err != nil {\n\t\t\t\treturn fmt.Errorf(\"external migration phase failed: %w\", err)\n\t\t\t}\n\t\t} else {\n\t\t\tmigrationCfg, err := ReadMigrationConfig(ipfsDir, \"\")\n\t\t\tif err != nil {\n\t\t\t\treturn fmt.Errorf(\"could not read migration config: %w\", err)\n\t\t\t}\n\n\t\t\t// Legacy migrations only support HTTPS downloads\n\t\t\tfetcher, err := GetMigrationFetcher(migrationCfg.DownloadSources, GetDistPathEnv(CurrentIpfsDist), nil)\n\t\t\tif err != nil {\n\t\t\t\treturn fmt.Errorf(\"failed to get migration fetcher: %w\", err)\n\t\t\t}\n\t\t\tdefer fetcher.Close()\n\n\t\t\tif err = RunMigration(ctx, fetcher, embeddedMigrationsMinVersion, ipfsDir, allowDowngrade); err != nil {\n\t\t\t\treturn fmt.Errorf(\"external migration phase failed: %w\", err)\n\t\t\t}\n\t\t}","sourceCodeStart":417,"sourceCodeEnd":453,"githubUrl":"https://github.com/ipfs/kubo/blob/329838acdfafae224582930457efe80aa217afc0/repo/fsrepo/migrations/migrations.go#L417-L453","documentation":"RunHybridMigrations upgrades an IPFS repo across the boundary of the hybrid scheme: external (downloaded binary) migrations are used below v16 and embedded Go migrations for v16+. When all required external migration binaries are found (or fetched) and runMigrationsFromPath executes them, any failure running those binaries is wrapped as \"external migration phase failed\". The error carries the underlying cause (binary exit status, download failure, incompatibility).","triggerScenarios":"Calling RunHybridMigrations(ctx, targetVer, ipfsDir, allowDowngrade) on the upgrade path where currentVer < 16 <= targetVer and foundAll==true, and runMigrationsFromPath returns an error: a migration binary fails to execute or exits non-zero, a needed migration binary cannot be run from the resolved path, or the underlying migration process is cancelled via ctx.","commonSituations":"Daemon startup (`ipfs daemon`) detects an old repo that must pass through the pre-v16 external migrations; the downloaded fs-repo-migrate binary fails on a corrupt or partially-migrated repo; a stale/partial migration binary left in the migrations dir; running out of disk space mid-migration; user Ctrl-C during a long migration.","solutions":["Re-run the migration (restarting the daemon) after freeing disk space and confirming the repo is intact; migrations are idempotent per version step","Run the failing migration manually with `ipfs fs-repo-migrations -to <ver>` or the fs-repo-migrate binary directly to see its full output","Delete partially written migration state/binaries under <ipfsDir>/fs-repo-migrations and retry","Check the repo with `ipfs repo fsck` / verify version file <ipfsDir>/version is consistent","Restore the repo from backup and migrate again if the datastore is corrupt"],"exampleFix":"// before (direct call that surfaces the opaque wrapped failure)\nerr := migrations.RunHybridMigrations(ctx, 16, ipfsPath, false)\n// after (pre-flight: check version and run external migrations manually for visibility)\nver, err := migrations.RepoVersion(ipfsPath)\nif err == nil && ver < 16 {\n\tif out, mErr := exec.Command(\"ipfs\", \"fs-repo-migrations\", \"-to\", \"16\").CombinedOutput(); mErr != nil {\n\t\tlog.Fatalf(\"external migration failed: %v: %s\", mErr, out)\n\t}\n}\nerr = migrations.RunHybridMigrations(ctx, 16, ipfsPath, false)","handlingStrategy":"validation","validationCode":"ver, err := migrations.RepoVersion(ipfsPath)\nif err != nil {\n\treturn fmt.Errorf(\"cannot read repo version: %w\", err)\n}\nif ver < 16 {\n\tif fi, err := os.Stat(filepath.Join(ipfsPath, \"fs-repo-migrations\")); err != nil || !fi.IsDir() {\n\t\treturn errors.New(\"pre-v16 repo: ensure fs-repo-migrate binaries are available before upgrading\")\n\t}\n}\n","typeGuard":"func isExternalMigrationPhaseFailed(err error) bool {\n\treturn err != nil && strings.Contains(err.Error(), \"external migration phase failed\")\n}\n","tryCatchPattern":"if err := migrations.RunHybridMigrations(ctx, targetVer, ipfsPath, false); err != nil {\n\tvar ctxErr error\n\tif errors.Is(ctx.Err(), context.Canceled) {\n\t\tctxErr = ctx.Err()\n\t}\n\tlog.Printf(\"external migration phase failed: %v (canceled=%v)\", err, ctxErr != nil)\n\treturn fmt.Errorf(\"repo migration interrupted; safe to retry: %w\", err)\n}\n","preventionTips":["Ensure adequate free disk space before starting the daemon on an old repo","Install or pre-stage fs-repo-migrate binaries instead of relying on runtime downloads","Avoid interrupting the daemon during migration; raise systemd TimeoutStartSec","Back up the repo directory before major version upgrades"],"tags":["go","migrations","repo-upgrade","subprocess"],"backgroundTag":"external-migration-binary-failed","analyzedSha":"329838acdfafae224582930457efe80aa217afc0","analyzedAt":"2026-09-03T18:30:52.135Z","contentChangedAt":"2026-09-03T18:30:52.135Z","schemaVersion":2},"datasetVersion":"2026-09-11T00:17:11.886Z"}