ipfs/kubo · critical

embedded migration phase failed: %w

Error message

embedded migration phase failed: %w

What it means

After the external phase completes, RunHybridMigrations runs phase 2: embedded (compiled-in) migrations from v16 up to the target version via RunEmbeddedMigrations. If any embedded migration step fails, the error is wrapped as "embedded migration phase failed". Unlike the external phase this requires no downloads, so failures almost always mean repo state, disk, or context cancellation problems.

Source

Thrown at repo/fsrepo/migrations/migrations.go:459

			}

			// Legacy migrations only support HTTPS downloads
			fetcher, err := GetMigrationFetcher(migrationCfg.DownloadSources, GetDistPathEnv(CurrentIpfsDist), nil)
			if err != nil {
				return fmt.Errorf("failed to get migration fetcher: %w", err)
			}
			defer fetcher.Close()

			if err = RunMigration(ctx, fetcher, embeddedMigrationsMinVersion, ipfsDir, allowDowngrade); err != nil {
				return fmt.Errorf("external migration phase failed: %w", err)
			}
		}

		// Phase 2: Use embedded migrations for v16+
		logger.Printf("Phase 2: Embedded migration from v%d to v%d", embeddedMigrationsMinVersion, targetVer)
		err = RunEmbeddedMigrations(ctx, targetVer, ipfsDir, allowDowngrade)
		if err != nil {
			return fmt.Errorf("embedded migration phase failed: %w", err)
		}

		logger.Printf("Hybrid migration completed successfully: v%d → v%d", currentVer, targetVer)
		return nil
	}

	// Case 4: Reverse hybrid migration (≥16 to <16)
	// Use embedded migrations for ≥16 steps, then external migrations for <16 steps
	logger.Printf("Starting reverse hybrid migration from version %d to %d", currentVer, targetVer)
	logger.Print("Using reverse hybrid migration strategy: embedded to v16, then external")

	// Phase 1: Use embedded migrations from current version down to v16 (if needed)
	if currentVer > embeddedMigrationsMinVersion {
		logger.Printf("Phase 1: Embedded downgrade from v%d to v%d", currentVer, embeddedMigrationsMinVersion)
		err = RunEmbeddedMigrations(ctx, embeddedMigrationsMinVersion, ipfsDir, allowDowngrade)
		if err != nil {
			return fmt.Errorf("embedded downgrade phase failed: %w", err)
		}

View on GitHub (pinned to 329838acdf)

Solutions

1. Re-run the migration — each version step is idempotent and resumes from the recorded repo version; check <ipfsDir>/version to see progress
2. Free disk space (migrations can temporarily need space comparable to the datastore) and retry
3. Increase the service timeout (e.g. systemd TimeoutStartSec) so long migrations are not killed
4. Back up the repo, then inspect the failing step's log output; run `ipfs repo fsck` if corruption is suspected

ver, err := migrations.RepoVersion(ipfsPath)
if err != nil {
	return err
}
if free, err := freeDisk(ipfsPath); err == nil && free < minRequiredBytes {
	return fmt.Errorf("insufficient disk for embedded migration v%d->v%d", ver, targetVer)
}

func isEmbeddedMigrationFailure(err error) bool {
	return err != nil && strings.Contains(err.Error(), "embedded migration phase failed")
}

if err := migrations.RunHybridMigrations(ctx, targetVer, ipfsPath, false); err != nil {
	if isEmbeddedMigrationFailure(err) && errors.Is(err, context.Canceled) {
		log.Print("embedded migration was interrupted; it is safe to re-run the daemon to resume")
		return err
	}
	return fmt.Errorf("embedded migration failed; check disk space and repo integrity: %w", err)
}

When it happens

Trigger: RunHybridMigrations upgrade path reaching RunEmbeddedMigrations(ctx, targetVer, ipfsDir, allowDowngrade) after external migrations succeed, and an embedded migration returns an error: datastore open/write failure, a migration-specific data error, disk full, or ctx cancelled mid-run.

Common situations: Long `ipfs daemon` startup on a large repo killed by Ctrl-C or systemd timeout mid-migration; disk full while rewriting the datastore; repo partially migrated by an earlier interrupted attempt; datastore engine misconfigured in config after upgrade.

Related errors

• external migration phase failed: %w
• embedded downgrade phase failed: %w
• cannot get migrations from unknown fetcher type
• unexpected keystore suffix %q, expected "0" or "1"
• creating datastore config for %s: %w

AI-assisted analysis of ipfs/kubo@329838acdf (2026-09-03). Data as JSON: /api/errors/bbe8434fe487fedf. Report an issue: GitHub.