dgraph-io/dgraph · error

failed to run in group streaming: %v

Error message

failed to run in group streaming: %v

What it means

streamInGroup runs the publisher, local subscriber, and forwarders in an errgroup. If any goroutine returns an error (publisher failure, local subscriber failure such as a Badger write error, etc.), eg.Wait returns it and streamInGroup wraps it as 'failed to run in group streaming: %v'. This is the top-level wrapper for any intra-group streaming failure on this node.

Source

Thrown at worker/import.go:626

			})
		}
	}

	eg.Go(func() error {
		defer ps.close()
		defer func() {
			if err := stream.Send(&api.StreamExtSnapshotResponse{}); err != nil {
				glog.Errorf("[import] failed to send close on in: %v", err)
			}
		}()
		if err := ps.handlePublisher(errGCtx, stream); err != nil {
			return fmt.Errorf("failed to run publisher: %v", err)
		}
		return nil
	})

	if err := eg.Wait(); err != nil {
		return fmt.Errorf("failed to run in group streaming: %v", err)
	}
	// Sends a StreamExtSnapshotResponse with the Finish flag set to true to indicate
	// the completion of the streaming process. If an error occurs during the send
	// operation, it is returned for further handling.
	if err := stream.Send(&api.StreamExtSnapshotResponse{Finish: true}); err != nil {
		glog.Errorf("failed to send done signal: %v", err)
	}

	// If this node is the leader and fails to reach a majority of nodes, we return an error.
	// This ensures that the data is reliably received by enough nodes before proceeding.
	if forward && !checkMajority(successfulNodes) {
		glog.Error("[import] Majority of nodes failed to receive data.")
		return fmt.Errorf("failed to send data to majority of the nodes")
	}
	return nil
}

func updateNodeStatus(ps *sync.RWMutex, successfulNodes map[string]bool, addr string, status bool) {

View on GitHub (pinned to 759e242be6)

Solutions

  1. Look earlier in the logs for '[import:flush] failed to run local subscriber' or 'failed to run forward subscriber' to find the root cause.
  2. Fix the underlying cause (disk space, network to peers, client stability) and retry the whole import.
  3. Verify all group members are alive and serving the group before starting the snapshot import.
  4. Ensure Badger storage has sufficient free space and healthy disks.

Example fix

null
Defensive patterns

Strategy: try-catch

Validate before calling

// pre-flight: all group members alive, disk OK, quorum possible
for _, m := range members { if m.AmDead || !reachable(m.Addr) { return fmt.Errorf("member %s not ready", m.Addr) } }

Try / catch

err := streamSnapshot(ctx)
if err != nil && strings.Contains(err.Error(), "failed to run in group streaming") {
    // scan logs for local subscriber / forwarder root cause, fix, then full retry
}

Prevention

When it happens

Trigger: Any member goroutine of the errgroup fails: handlePublisher error, runLocalSubscriber error (e.g. Badger write failure, downstream ack error relayed through pipeTwoStream), or ctx cancellation of the stream.

Common situations: Disk full on the follower writing to Badger; one group member disconnected so the leader's forwarder aborts; client canceled the import; gRPC stream reset mid-import.

Related errors


AI-assisted analysis of dgraph-io/dgraph@759e242be6 (2026-09-01). Data as JSON: /api/errors/3256a6fc74202df1. Report an issue: GitHub.