dgraph-io/dgraph · error

stream orchestration failed for group [%v]: %w, badger path:

Error message

stream orchestration failed for group [%v]: %w, badger path: %s

What it means

Returned by streamBadger when badger's Stream.Orchestrate(ctx) fails while iterating the pdir and pushing packets through the Send callback. The message includes the badger directory path to help locate the offending pdir. Orchestrate aggregates errors from stream.Send (chunk send/recv) or internal badger iteration failures.

Source

Thrown at dgraph/cmd/dgraphimport/import_client.go:234

	stream := ps.NewStreamAt(math.MaxUint64)
	stream.LogPrefix = "[import] Sending external snapshot to group [" + fmt.Sprintf("%d", groupId) + "]"
	stream.KeyToList = nil
	stream.Send = func(buf *z.Buffer) error {
		p := &api.StreamPacket{Data: buf.Bytes()}
		if err := out.Send(&api.StreamExtSnapshotRequest{Pkt: p}); err != nil && !errors.Is(err, io.EOF) {
			return fmt.Errorf("failed to send data chunk: %w", err)
		}
		if _, err := out.Recv(); err != nil {
			return fmt.Errorf("failed to receive response for group ID [%v] from the server: %w", groupId, err)
		}
		glog.Infof("[import] Group [%v]: Received ACK for sending data chunk", groupId)

		return nil
	}

	// Execute the stream process
	if err := stream.Orchestrate(ctx); err != nil {
		return fmt.Errorf("stream orchestration failed for group [%v]: %w, badger path: %s", groupId, err, ps.Opts().Dir)
	}

	// Send the final 'done' signal to mark completion
	glog.Infof("[import] Sending completion signal for group [%d]", groupId)
	done := &api.StreamPacket{Done: true}

	if err := out.Send(&api.StreamExtSnapshotRequest{Pkt: done}); err != nil && !errors.Is(err, io.EOF) {
		return fmt.Errorf("failed to send 'done' signal for group [%d]: %w", groupId, err)
	}

	for {
		if ctx.Err() != nil {
			return ctx.Err()
		}
		resp, err := out.Recv()
		if errors.Is(err, io.EOF) {
			return fmt.Errorf("server closed stream before Finish=true for group [%d]", groupId)
		}

View on GitHub (pinned to 759e242be6)

Solutions

1. Note the badger path in the message and check that pdir for IO errors/corruption (badger check / open read-only).
2. Unwrap the inner error to see whether it came from the Send callback (network) or badger iteration (disk).
3. If disk-related, re-export the pdir from a healthy Alpha.
4. If network-related, fix connectivity and re-run the group's stream.
5. Give the context enough deadline for the full directory size.

Example fix

// before
if err := stream.Orchestrate(ctx); err != nil {
	return fmt.Errorf("stream orchestration failed for group [%v]: %w, badger path: %s", groupId, err, ps.Opts().Dir)
}
// after
if err := stream.Orchestrate(ctx); err != nil {
	if ctx.Err() != nil {
		return fmt.Errorf("orchestration for group %d timed out (dir %s): %w", groupId, ps.Opts().Dir, ctx.Err())
	}
	return fmt.Errorf("stream orchestration failed for group [%v]: %w, badger path: %s", groupId, err, ps.Opts().Dir)
}

db, err := badger.OpenManaged(badger.DefaultOptions(pdir).WithReadOnly(true).WithNumVersionsToKeep(math.MaxInt32))
if err != nil {
	return fmt.Errorf("pdir %s failed integrity precheck: %w", pdir, err)
}
it := db.NewStreamAt(math.MaxUint64)
_ = it.ChooseKey = nil // ensure stream can be built
_ = db.Close()

if err := stream.Orchestrate(ctx); err != nil {
	if errors.Is(err, badger.ErrDBClosed) || isIOError(err) {
		return fmt.Errorf("re-export pdir %s; source data unreadable", ps.Opts().Dir)
	}
	return fmt.Errorf("stream orchestration failed for group [%v]: %w, badger path: %s", groupId, err, ps.Opts().Dir)
}

When it happens

Trigger: Badger iteration error while reading the pdir (corrupted file, IO error); the Send callback returned an error (chunk send/recv failures, errors 86/87); ctx cancelled mid-orchestration.

Common situations: Reading a corrupted or truncated export directory; disk read errors on the import host; transfer interrupted by context timeout on very large pdirs.

Related errors

• failed to open BadgerDB at [%v]: %v
• badger streaming failed for group [%v]: %v
• failed to start external snapshot stream for group %d: %w
• failed to send request for group ID [%v] to the server: %w
• failed to receive response for group ID [%v] from the server

AI-assisted analysis of dgraph-io/dgraph@759e242be6 (2026-09-01). Data as JSON: /api/errors/39e687e1d792e9ce. Report an issue: GitHub.