dgraph-io/dgraph · error

While proposing

Error message

While proposing

What it means

This error wraps a failure returned by the Raft Raft().Propose() call (pkg/errors.Wrapf), meaning a proposal (e.g. a mutation) could not be submitted to the Raft group within the given timeout. It is thrown in the proposal path in worker/proposal.go after a tracing 'Proposing' event, so the underlying cause is whatever Propose returned (leadership loss, deadline exceeded, shard unavailable).

Source

Thrown at worker/proposal.go:238

		// We don't need to extend from base ctx as it might have a timeout. This timeout
		// is only to find the proposal back via Raft.
		cctx, cancel := context.WithCancel(context.Background())
		defer cancel()

		errCh := make(chan error, 1)
		pctx := &conn.ProposalCtx{
			ErrCh: errCh,
			Ctx:   cctx,
		}
		x.AssertTruef(n.Proposals.Store(key, pctx), "Found existing proposal with key: [%x]", key)
		defer n.Proposals.Delete(key) // Ensure that it gets deleted on return.

		span.AddEvent("Proposing", trace.WithAttributes(
			attribute.Int64("key", int64(key)),
			attribute.String("timeout", timeout.String())))

		if err = n.Raft().Propose(cctx, data); err != nil {
			return errors.Wrapf(err, "While proposing")
		}

		timer := time.NewTimer(timeout)

		for {
			select {
			case err = <-errCh:
				// We arrived here by a call to n.Proposals.Done().
				return err
			case <-ctx.Done():
				glog.Warningf("Context expired while processing proposal %v", ctx.Err())
				return ctx.Err()
			case <-timer.C:
				if atomic.LoadUint32(&pctx.Found) > 0 {
					// We found the proposal in CommittedEntries. No need to retry.
				} else {
					span.AddEvent("Timeout reached", trace.WithAttributes(
						attribute.String("timeout", timeout.String())))

View on GitHub (pinned to 759e242be6)

Solutions

  1. Retry the mutation — if this node lost leadership, a new leader will accept the proposal
  2. Increase the client-side mutation/operation timeout and retry with backoff
  3. Check Raft group health (leader elections in logs, /health endpoint) and ensure quorum of Alphas is up
  4. Inspect the wrapped cause with errors.Cause()/Unwrap to address the root error

Example fix

// before
err := n.Raft().Propose(ctx, data)
if err != nil { return err }
// after
err := n.Raft().Propose(ctx, data)
if err != nil {
    if isLeadershipOrTimeout(err) {
        time.Sleep(backoff)
        err = n.Raft().Propose(ctx, data)
    }
    return err
}
Defensive patterns

Strategy: retry

Validate before calling

// Prefer a healthy, leader-connected client before proposing
if !client.Healthy(ctx) {
    return fmt.Errorf("alpha unreachable; pick another client")
}
ctx, cancel := context.WithTimeout(ctx, 30*time.Second)
defer cancel()

Try / catch

err := propose(ctx, data)
var netErr net.Error
if errors.As(err, &netErr) || isContextDeadline(err) || isNotLeader(err) {
    time.Sleep(backoff)
    err = propose(ctx, data)
}
if err != nil {
    log.Printf("proposal failed: %v", err)
}

Prevention

When it happens

Trigger: Calling n.Raft().Propose(cctx, data) with a context that times out or is cancelled before the proposal is committed; losing Raft leadership mid-proposal; the Raft group being unavailable (quorum lost, node restarting).

Common situations: Heavy write load or slow disks making Raft commits exceed the proposal timeout; an Alpha that was just demoted from leader still serving mutations; network partitions between Alpha nodes; client set a short mutation timeout.

Related errors


AI-assisted analysis of dgraph-io/dgraph@759e242be6 (2026-09-01). Data as JSON: /api/errors/78a938cebd7d0b2f. Report an issue: GitHub.