ipfs/kubo · critical

external downgrade phase failed: %w

Error message

external downgrade phase failed: %w

What it means

On the downgrade path, when all required external migration binaries are found locally, RunHybridMigrations executes them via runMigrationsFromPath with downgrade=true. If any external downgrade migration fails to run or exits non-zero, the error is wrapped as "external downgrade phase failed". This is the downgrade counterpart of error 690.

Source

Thrown at repo/fsrepo/migrations/migrations.go:500

		logger.Printf("Phase 2: External downgrade from v%d to v%d", embeddedMigrationsMinVersion, targetVer)

		// Check for external migration binaries in PATH first
		migrations, binPaths, err := findMigrations(ctx, embeddedMigrationsMinVersion, targetVer)
		if err != nil {
			return fmt.Errorf("could not determine external migration paths: %w", err)
		}

		foundAll := true
		for _, migName := range migrations {
			if _, exists := binPaths[migName]; !exists {
				foundAll = false
				break
			}
		}

		if foundAll {
			if err = runMigrationsFromPath(ctx, migrations, binPaths, ipfsDir, logger, true); err != nil {
				return fmt.Errorf("external downgrade phase failed: %w", err)
			}
		} else {
			migrationCfg, err := ReadMigrationConfig(ipfsDir, "")
			if err != nil {
				return fmt.Errorf("could not read migration config: %w", err)
			}

			// Legacy migrations only support HTTPS downloads
			fetcher, err := GetMigrationFetcher(migrationCfg.DownloadSources, GetDistPathEnv(CurrentIpfsDist), nil)
			if err != nil {
				return fmt.Errorf("failed to get migration fetcher: %w", err)
			}
			defer fetcher.Close()

			if err = RunMigration(ctx, fetcher, targetVer, ipfsDir, allowDowngrade); err != nil {
				return fmt.Errorf("external downgrade phase failed: %w", err)
			}
		}

View on GitHub (pinned to 329838acdf)

Solutions

1. Back up the repo, then re-run the downgrade after fixing the underlying cause printed by the wrapped error
2. Run the failing binary directly (e.g. <ipfsDir>/fs-repo-migrations/fs-repo-migrate-15-14 -path=<ipfsDir>) to see its full output
3. Delete stale migration binaries from <ipfsDir>/fs-repo-migrations and re-obtain the correct versions
4. If the newer schema's data is not representable in the older version, export pins/IPNS keys and accept data loss, or keep the newer binary instead

Example fix

// before (blind retry of failing downgrade)
err := migrations.RunHybridMigrations(ctx, 14, ipfsPath, true)
// after (inspect the failing step manually first)
out, mErr := exec.Command(filepath.Join(ipfsPath, "fs-repo-migrations", "fs-repo-migrate-15-14"), "-path="+ipfsPath).CombinedOutput()
log.Printf("migration output: %s", out)
if mErr == nil {
	err = migrations.RunHybridMigrations(ctx, 14, ipfsPath, true)
}

for _, bin := range requiredBins {
	path := filepath.Join(ipfsPath, "fs-repo-migrations", bin)
	if fi, err := os.Stat(path); err != nil || fi.IsDir() {
		return fmt.Errorf("missing downgrade binary %s", bin)
	}
	if err := exec.Command(path, "--help").Run(); err != nil {
		return fmt.Errorf("downgrade binary %s not executable: %w", bin, err)
	}
}

func isExternalDowngradeFailure(err error) bool {
	return err != nil && strings.Contains(err.Error(), "external downgrade phase failed")
}

if err := migrations.RunHybridMigrations(ctx, targetVer, ipfsPath, true); err != nil {
	if isExternalDowngradeFailure(err) {
		log.Printf("downgrade failed; restore from backup at %s.bak if repo state is inconsistent", ipfsPath)
		return err
	}
	return err
}

When it happens

Trigger: RunHybridMigrations downgrade path (targetVer < 16, allowDowngrade==true) with foundAll==true and runMigrationsFromPath(ctx, migrations, binPaths, ipfsDir, logger, true) returning an error: a fs-repo-migrate-N binary fails to start or exits non-zero while rolling the repo back from v16 to targetVer.

Common situations: Rolling a repo back to an older kubo release after a failed upgrade; stale or version-mismatched migration binaries left in <ipfsDir>/fs-repo-migrations; downgrade migration aborts because the newer schema holds data it cannot represent; interrupted earlier downgrade left inconsistent state.

Related errors

• downgrade not allowed from %d to %d
• external migration phase failed: %w
• embedded downgrade phase failed: %w
• abort failed: close: %w, remove: %v
• source field %s does not exist

AI-assisted analysis of ipfs/kubo@329838acdf (2026-09-03). Data as JSON: /api/errors/e36733b320ce4e1a. Report an issue: GitHub.