dgraph-io/dgraph · error

operation %s is already running

Error message

operation %s is already running

What it means

Raised in startTaskAtTs (worker/draft.go:173) when a new indexing operation conflicts with an existing operation. Indexing cancels rollups, is blocked by a backup running at a lower (earlier) timestamp, and fails against any other running operation kind (restore, snapshot, predmove) via the default branch.

Source

Thrown at worker/draft.go:173

		// only one backup operation should be active any given moment. Also, indexing at higher
		// timestamp can also run concurrently with backup.
		for otherId, otherOp := range n.ops {
			if otherId == opBackup {
				return nil, errors.Errorf("another backup operation is already running")
			}
			// Remove from map and signal the closer to cancel the operation.
			delete(n.ops, otherId)
			otherOp.SignalAndWait()
		}
	case opIndexing:
		for otherId, otherOp := range n.ops {
			switch otherId {
			case opBackup:
				if otherOp.ts < ts {
					// If backup is running at higher timestamp, then indexing can't be executed.
					continue
				}
				return nil, errors.Errorf("operation %s is already running", otherId)
			case opRollup:
				// Remove from map and signal the closer to cancel the operation.
				delete(n.ops, otherId)
				otherOp.SignalAndWait()
			default:
				return nil, errors.Errorf("operation %s is already running", otherId)
			}
		}
	case opSnapshot, opPredMove:
		for otherId, otherOp := range n.ops {
			if otherId == opRollup {
				// Remove from map and signal the closer to cancel the operation.
				delete(n.ops, otherId)
				otherOp.SignalAndWait()
			} else {
				return nil, errors.Errorf("operation %s is already running", otherId)
			}
		}

View on GitHub (pinned to 759e242be6)

Solutions

  1. Wait for the conflicting operation (backup/restore/snapshot) to complete, then re-issue the index build.
  2. If a backup at an older timestamp blocks indexing, wait for it to finish — the message names the operation via %s.
  3. Retry the schema alter after the background operation finishes; schema changes are idempotent.
  4. Schedule schema/index changes outside backup windows.

Example fix

// before: alter immediately during backup
client.Alter(schemaWithIndex)
// after
if err := client.Alter(schemaWithIndex); err != nil && strings.Contains(err.Error(), "is already running") {
    waitForOpsToFinish()
    client.Alter(schemaWithIndex)
}
Defensive patterns

Strategy: retry

Validate before calling

waitForNoRunningOps(cluster, ["backup","restore","snapshot"]) then applyIndexAlter()

Try / catch

if err := client.Alter(schema); err != nil && strings.Contains(err.Error(), "is already running") {
    <-opsFinished
    return client.Alter(schema)
}

Prevention

When it happens

Trigger: Adding an index (AlterRequest with DropOp/IndexAttr) that triggers startTask(opIndexing) while n.ops holds a backup with otherOp.ts < ts, or any operation other than rollup/backup.

Common situations: Applying schema changes while a backup or restore runs; building several indexes concurrently on a busy cluster; snapshot streaming overlapping with index creation.

Related errors


AI-assisted analysis of dgraph-io/dgraph@759e242be6 (2026-09-01). Data as JSON: /api/errors/40ac20e5b1512a84. Report an issue: GitHub.