What causes "too many pending tasks, please try again later" in dgraph-io/dgraph?

Retry with exponential backoff — the error explicitly advises trying later

too many pending tasks, please try again later

Error message

too many pending tasks, please try again later

What it means

enqueue pushes new tasks onto a bounded buffered channel t.queue; when the channel is full, the select falls to the default branch and rejects the task with 'too many pending tasks, please try again later'. The queue runs one task at a time, so a burst of submissions or a stuck task fills the buffer.

Source

Thrown at worker/queue.go:189

	default:
		panic(fmt.Sprintf("invalid TaskKind: %d", kind))
	}

	t.logMu.Lock()
	defer t.logMu.Unlock()

	task := taskRequest{
		id:  t.newId(),
		req: req,
	}
	select {
	// t.logMu must be acquired before pushing to t.queue, otherwise the worker might start the
	// task, and won't be able to find it in t.log.
	case t.queue <- task:
		t.log.Set(task.id, newTaskMeta(kind, TaskStatusQueued).uint64())
		return task.id, nil
	default:
		return 0, fmt.Errorf("too many pending tasks, please try again later")
	}
}

// get retrieves metadata for a given task ID.
func (t *tasks) get(id uint64) (TaskMeta, error) {
	if t == nil {
		return 0, fmt.Errorf("task queue hasn't been initialized yet")
	}

	if id == 0 || id == math.MaxUint64 {
		return 0, fmt.Errorf("task ID is invalid: %d", id)
	}
	t.logMu.Lock()
	defer t.logMu.Unlock()
	meta := TaskMeta(t.log.Get(id))
	if meta == 0 {
		return 0, fmt.Errorf("task does not exist or has expired")
	}

View on GitHub (pinned to 759e242be6)

Solutions

Retry with exponential backoff — the error explicitly advises trying later
Serialize task submissions client-side or add a distributed lock/queue in front of the cluster
Investigate why the running task is slow/hung (long backup, storage latency) and cancel/fix it
Reduce frequency of scheduled backups/exports or cluster-shard the workload

Example fix

// before
id, err := tasks.Enqueue(req)
if err != nil { return err }
// after
var id uint64
var err error
for attempt := 0; attempt < 5; attempt++ {
    id, err = tasks.Enqueue(req)
    if err == nil || !strings.Contains(err.Error(), "too many pending tasks") { break }
    time.Sleep(time.Duration(1<<attempt) * time.Second)
}

Defensive patterns

Strategy: retry

Validate before calling

// Check current backlog before submitting (if exposed)
// or simply rate-limit client-side:
if inflight >= maxInflight { return fmt.Errorf("local submission limit reached") }

Try / catch

var id uint64
var err error
for i := 0; i < 5; i++ {
    id, err = tasks.Enqueue(req)
    if err == nil || !strings.Contains(err.Error(), "too many pending tasks") { break }
    time.Sleep(time.Duration(1<<i) * time.Second)
}

Prevention

Throttle and serialize task submissions client-side
Monitor for long-running tasks backing up the single-worker queue
Alert on repeated 'too many pending tasks' occurrences
Schedule backups/exports to avoid overlap

When it happens

Trigger: Calling Enqueue when t.queue already holds its maximum number of pending tasks — many queued backups/exports while the single worker is blocked on a long-running task.

Common situations: Automated backup schedules piling up because one backup is hung on slow storage; multiple clients submitting exports concurrently; the worker stuck on an I/O-bound task causing backlog growth.

Related errors

AI-assisted analysis of dgraph-io/dgraph@759e242be6 (2026-09-01). Data as JSON: /api/errors/8b0f809a38d429af. Report an issue: GitHub.