dgraph-io/dgraph · error

badger streaming failed for group [%v]: %v

Error message

badger streaming failed for group [%v]: %v

What it means

Wrapper returned by streamSnapshotForGroup when streamBadger fails: Badger's Stream.Orchestrate, per-chunk send/recv, or the final done-signal exchange errored. It records the group ID so operators know which group's pdir stream aborted.

Source

Thrown at dgraph/cmd/dgraphimport/import_client.go:207

		}
	}()

	// Send group ID as the first message in the stream
	glog.Infof("[import] Sending request for streaming external snapshot for group ID [%v]", groupId)
	groupReq := &api.StreamExtSnapshotRequest{GroupId: groupId}
	if err := out.Send(groupReq); err != nil {
		return fmt.Errorf("failed to send request for group ID [%v] to the server: %w", groupId, err)
	}
	if _, err := out.Recv(); err != nil {
		return fmt.Errorf("failed to receive response for group ID [%v] from the server: %w", groupId, err)
	}

	glog.Infof("[import] Group [%v]: Received ACK for sending group request", groupId)

	// Configure and start the BadgerDB stream
	glog.Infof("[import] Starting BadgerDB stream for group [%v]", groupId)
	if err := streamBadger(ctx, ps, out, groupId); err != nil {
		return fmt.Errorf("badger streaming failed for group [%v]: %v", groupId, err)
	}
	return nil
}

// streamBadger runs a BadgerDB stream to send key-value pairs to the specified group.
// It creates a new stream at the maximum sequence number and sends the data to the specified group.
// It also sends a final 'done' signal to mark completion.
func streamBadger(ctx context.Context, ps *badger.DB, out api.Dgraph_StreamExtSnapshotClient, groupId uint32) error {
	stream := ps.NewStreamAt(math.MaxUint64)
	stream.LogPrefix = "[import] Sending external snapshot to group [" + fmt.Sprintf("%d", groupId) + "]"
	stream.KeyToList = nil
	stream.Send = func(buf *z.Buffer) error {
		p := &api.StreamPacket{Data: buf.Bytes()}
		if err := out.Send(&api.StreamExtSnapshotRequest{Pkt: p}); err != nil && !errors.Is(err, io.EOF) {
			return fmt.Errorf("failed to send data chunk: %w", err)
		}
		if _, err := out.Recv(); err != nil {
			return fmt.Errorf("failed to receive response for group ID [%v] from the server: %w", groupId, err)

View on GitHub (pinned to 759e242be6)

Solutions

1. Unwrap the message to find the inner stage (orchestration vs send/recv vs done signal) and its gRPC/badger cause.
2. Check Alpha logs and health around the failure time (OOM, restart, disk full on server).
3. Re-run the import for that group; snapshot streaming is all-or-nothing per group.
4. Increase the context deadline to cover the full transfer of the pdir.
5. Validate the pdir's badger DB integrity before streaming (open read-only and iterate a few keys).

Example fix

// before
if err := streamBadger(ctx, ps, out, groupId); err != nil {
	return fmt.Errorf("badger streaming failed for group [%v]: %v", groupId, err)
}
// after
if err := streamBadger(ctx, ps, out, groupId); err != nil {
	if ctx.Err() != nil {
		return fmt.Errorf("badger streaming for group %d cancelled/timed out: %w", groupId, ctx.Err())
	}
	return fmt.Errorf("badger streaming failed for group [%v]: %v", groupId, err)
}

db, err := badger.OpenManaged(badger.DefaultOptions(pdir).WithReadOnly(true))
if err != nil {
	return fmt.Errorf("precheck: pdir unusable: %w", err)
}
_ = db.Close()

if err := streamBadger(ctx, ps, out, groupId); err != nil {
	log.Printf("group %d stream failed (%v); scheduling full retry after fixing cause", groupId, err)
	return scheduleGroupImportRetry(ctx, dc, pdir, groupId)
}

When it happens

Trigger: Any failure inside streamBadger: stream.Orchestrate error, Send/Recv failures per chunk, done-signal send failure, or context cancellation during the loop — all surfaced through this wrap.

Common situations: Long-running stream killed by network interruption mid-transfer; Alpha OOM/crash while receiving packets; context cancelled because import deadline too small for the dataset size; badger read errors from a corrupted export directory.

Related errors

• failed to start external snapshot stream for group %d: %w
• failed to send request for group ID [%v] to the server: %w
• failed to receive response for group ID [%v] from the server
• failed to send data chunk: %w
• stream orchestration failed for group [%v]: %w, badger path:

AI-assisted analysis of dgraph-io/dgraph@759e242be6 (2026-09-01). Data as JSON: /api/errors/0c6912aff83c73e0. Report an issue: GitHub.