dgraph-io/dgraph · error

another backup operation is already running

Error message

another backup operation is already running

What it means

Raised in startTaskAtTs (worker/draft.go:159) when a backup is requested while another backup is already running. Backup cancels all other operation kinds but is not allowed to run concurrently with another backup.

Solutions

  1. Wait for the running backup to finish; check its status first.
  2. Widen the backup schedule interval so it exceeds the longest observed backup duration.
  3. Add a distributed lock / job queue so only one backup request is ever in flight.
  4. Restart the node if a stale backup entry is blocking new backups.

Example fix

// before
cron.AddFunc("0 * * * *", doBackup)
// after: guard with in-flight check
if !backupInFlight.Load() {
    backupInFlight.Store(true)
    defer backupInFlight.Store(false)
    doBackup()
}
Defensive patterns

Strategy: validation

Validate before calling

if backupRunning(cluster) { return; }
scheduleBackup()

Prevention

When it happens

Trigger: Calling startTask(opBackup) (POST /backup or backup gRPC) while a previous backup is still tracked in n.ops for the group.

Common situations: Overlapping scheduled backups (e.g. hourly cron on clusters where a backup takes longer than an hour); manual backup triggered while automated backup is running.

Related errors


AI-assisted analysis of dgraph-io/dgraph@759e242be6 (2026-09-01). Data as JSON: /api/errors/68e3d9d1858af278. Report an issue: GitHub.

Appendix: source

Thrown at worker/draft.go:159

		go posting.IncrRollup.Process(closer, State.GetTimestamp)
	case opRestore:
		// Restores cancel all other operations, except for other restores since
		// only one restore operation should be active any given moment.
		for otherId, otherOp := range n.ops {
			if otherId == opRestore {
				return nil, errors.Errorf("another restore operation is already running")
			}
			// Remove from map and signal the closer to cancel the operation.
			delete(n.ops, otherId)
			otherOp.SignalAndWait()
		}
	case opBackup:
		// Backup cancels all other operations, except for other backups since
		// only one backup operation should be active any given moment. Also, indexing at higher
		// timestamp can also run concurrently with backup.
		for otherId, otherOp := range n.ops {
			if otherId == opBackup {
				return nil, errors.Errorf("another backup operation is already running")
			}
			// Remove from map and signal the closer to cancel the operation.
			delete(n.ops, otherId)
			otherOp.SignalAndWait()
		}
	case opIndexing:
		for otherId, otherOp := range n.ops {
			switch otherId {
			case opBackup:
				if otherOp.ts < ts {
					// If backup is running at higher timestamp, then indexing can't be executed.
					continue
				}
				return nil, errors.Errorf("operation %s is already running", otherId)
			case opRollup:
				// Remove from map and signal the closer to cancel the operation.
				delete(n.ops, otherId)
				otherOp.SignalAndWait()

View on GitHub (pinned to 759e242be6)