dgraph-io/dgraph · critical

RunRestore failed to reduce

Error message

RunRestore failed to reduce

What it means

This error wraps a failure from RunReducer, which replays the mapped backup files from the temp map directory into the Badger StreamWriter during offline restore. It means the reduce phase (reading sorted map output and streaming KVs into the DB) failed, so the group's post directory is incomplete and the restore is aborted.

Source

Thrown at worker/online_restore.go:603

			WithCompression(ctype).
			WithZSTDCompressionLevel(clevel).
			WithSyncWrites(false).
			WithBlockCacheSize(100 * (1 << 20)).
			WithIndexCacheSize(100 * (1 << 20)).
			WithNumVersionsToKeep(math.MaxInt32).
			WithEncryptionKey(key).
			WithNamespaceOffset(x.NamespaceOffset))
		if err != nil {
			return LoadResult{Err: errors.Wrap(err, "RunRestore failed to open DB")}
		}
		defer db.Close()

		sw := db.NewStreamWriter()
		if err := sw.Prepare(); err != nil {
			return LoadResult{Err: errors.Wrap(err, "while preparing DB")}
		}
		if err := RunReducer(sw, mapDir); err != nil {
			return LoadResult{Err: errors.Wrap(err, "RunRestore failed to reduce")}
		}
		if err := sw.Flush(); err != nil {
			return LoadResult{Err: errors.Wrap(err, "while stream writer flush")}
		}
		if err := x.WriteGroupIdFile(pdir, gid); err != nil {
			return LoadResult{Err: errors.Wrap(err, "RunRestore failed to write group id file")}
		}
	}
	// TODO: Fix this return value.
	return LoadResult{Version: manifest.ValidReadTs()}
}

func buildPredsForDefaultNamespace(restorePreds []string, fromNamespace uint64) []string {
	filtered := restorePreds[:0]
	for _, pred := range restorePreds {
		ns, attr := x.ParseNamespaceAttr(pred)
		if fromNamespace == ns {
			// update namespace value to 0

View on GitHub (pinned to 759e242be6)

Solutions

1. Retry the restore to rule out transient disk/TMPDIR issues; do not delete the temp map dir while restoring.
2. Point TMPDIR to a large local disk (e.g. --tmp with ample space) for big restores.
3. Verify the backup files are complete: re-download/copy from the backup location and check the manifest.
4. Confirm the correct --backup_id matching the target DB state is used.
5. Look at the wrapped inner error in logs to identify the specific KV or map file that failed.

if err := verifyBackupManifest(location, backupId); err != nil {
	return fmt.Errorf("backup files incomplete: %v", err)
}
ensureDiskFree(os.TempDir(), mapSpaceRequired)

if err := RunOfflineRestore(...); strings.Contains(err.Error(), "failed to reduce") {
	// verify backup integrity, enlarge TMPDIR, retry from clean state
}

When it happens

Trigger: RunOfflineRestore fails during RunReducer(sw, mapDir): the temp 'restore-map' directory was deleted or corrupted mid-restore, backup files are truncated/corrupt, the map output fails checksums (wrong --backup_id or mixing backup generations), or the StreamWriter rejects a write (e.g. invalid/oversized key, encryption mismatch).

Common situations: TMPDIR/tmp volume too small and cleaned during restore; selecting the wrong --backup_id from a multi-backup location; copying a backup directory incompletely between machines; interrupted network transfer corrupting backup files.

Related errors

• Error while getting cors from db.
• another restore operation is already running
• Pending transactions found. Please retry operation
• while retrieving manifests
• cannot wait for restore ts %d

AI-assisted analysis of dgraph-io/dgraph@759e242be6 (2026-09-01). Data as JSON: /api/errors/f5af01e812f49565. Report an issue: GitHub.