apache/beam · error

failed to reprocess with merged windows, got :%v

Error message

failed to reprocess with merged windows, got :%v

What it means

After successfully merging session windows in FinishBundle, the CoGBK node reprocesses its buffered values under the merged windows via reprocessByWindow. If that reprocessing step fails, the bundle is failed with 'failed to reprocess with merged windows, got :%v'. It indicates the merged-window replay could not complete for the buffered data.

Source

Thrown at sdks/go/pkg/beam/runners/direct/gbk.go:108

	if !ok {
		g = &group{
			key:    exec.FullValue{Elm: elm.Elm, Timestamp: elm.Timestamp, Windows: ws},
			values: make([][]exec.FullValue, len(n.Edge.Input)),
		}
		m[key] = g
	}
	return g, nil
}

func (n *CoGBK) FinishBundle(ctx context.Context) error {
	winKind := n.Edge.Input[0].From.WindowingStrategy().Fn.Kind
	if winKind == window.Sessions {
		mergeMap, mergeErr := n.mergeWindows()
		if mergeErr != nil {
			return errors.Errorf("failed to merge windows, got: %v", mergeErr)
		}
		if reprocessErr := n.reprocessByWindow(mergeMap); reprocessErr != nil {
			return errors.Errorf("failed to reprocess with merged windows, got :%v", reprocessErr)
		}
	}
	for key, g := range n.m {
		values := make([]exec.ReStream, len(g.values))
		for i, list := range g.values {
			values[i] = &exec.FixedReStream{Buf: list}
		}
		if err := n.Out.ProcessElement(ctx, &g.key, values...); err != nil {
			return err
		}
		delete(n.m, key)
	}
	return n.Out.FinishBundle(ctx)
}

func (n *CoGBK) mergeWindows() (map[typex.Window]int, error) {
	sort.Slice(n.wins, func(i int, j int) bool {
		return n.wins[i].MaxTimestamp() < n.wins[j].MaxTimestamp()

View on GitHub (pinned to 12126d8942)

Solutions

1. Read the embedded reprocessErr to pinpoint which key/window failed replay.
2. Ensure all inputs to the CoGBK share the same windowing strategy and coder; mixed windows confuse the merge map.
3. Switch to a production runner (Dataflow/Flink) for session-windowed CoGBK workloads — the direct runner is for testing only.
4. Update the Beam Go SDK to the latest version to pick up fixes to the CoGBK session-reprocessing path.

Example fix

// before
w1 := beam.WindowInto(s, window.NewSessions(time.Minute), colA)
w2 := beam.WindowInto(s, window.NewFixedWindows(time.Hour), colB) // mismatched strategies into same CoGBK
// after
w1 := beam.WindowInto(s, window.NewSessions(time.Minute), colA)
w2 := beam.WindowInto(s, window.NewSessions(time.Minute), colB)

// ensure all CoGBK inputs share the same windowing strategy
for _, col := range inputs {
    if col.WindowingStrategy().Fn.Kind != window.Sessions {
        return fmt.Errorf("CoGBK input must use session windows")
    }
}

err := plan.Execute(ctx, "", exec.DataContext{})
if err != nil && strings.Contains(err.Error(), "failed to reprocess with merged windows") {
    log.Printf("session replay failed; align windowing across CoGBK inputs: %v", err)
}

When it happens

Trigger: FinishBundle on a session-windowed CoGBK node where mergeWindows succeeded but reprocessByWindow(mergeMap) returns an error — e.g. a buffered key/window missing from the merge map or an internal stream-construction failure during replay.

Common situations: Session windows with many merged groups; pipelines where elements arrive in windows not represented in the merge result; SDK bugs in the direct runner's reprocessing logic; very large buffered bundles exhausting assumptions of the replay path.

Understand the failure class

Background: "This is a bug, please report it": internal invariant violations, unreachable panics, and SNH errors explained — this error's family across 47 libraries.

Related errors

• failed to merge windows, got: %v
• failed getGroup for %v: %v
• error encoding pane %v: non-speculative index value must be
• failed to map main input window to side input window with Wi
• mismatched CoGBK windowing strategies: %v, want %v

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/8c1e6c15cafcf107. Report an issue: GitHub.