DoFn terminated without fully processing restriction

Error message

DoFn terminated without fully processing restriction

What it means

For splittable DoFns, processSingleWindow expects the DoFn to have claimed and fully processed its restriction before returning. rtErrHelper is called when ProcessElement returns a nil error but the restriction tracker reports unfinished work (TryClaim not exhausted), so Beam raises this error instead of silently treating the restriction as done.

Source

Thrown at sdks/go/pkg/beam/core/runtime/exec/pardo.go:223

		// We do not forward a ProcessContinuation on its own
		if val.Elm == nil {
			return val.Continuation, nil
		}
		return val.Continuation, n.Out[0].ProcessElement(n.ctx, val)
	}

	if mainIn.RTracker != nil && !mainIn.RTracker.IsDone() {
		return nil, rtErrHelper(mainIn.RTracker.GetError())
	}

	return nil, nil
}

func rtErrHelper(err error) error {
	if err != nil {
		return err
	}
	return errors.New("DoFn terminated without fully processing restriction")
}

// mustExplodeWindows returns true iif we need to call the function
// for each window. It is needed if the function either observes the
// window, either directly or indirectly via (windowed) side inputs or state.
func mustExplodeWindows(fn *funcx.Fn, elm *FullValue, usesSideInput bool) bool {
	if len(elm.Windows) < 2 {
		return false
	}
	_, explode := fn.Window()
	_, observesState := fn.StateProvider()
	return explode || usesSideInput || observesState
}

// FinishBundle does post-bundle processing operations for the DoFn.
// Note: This is not a "FinalizeBundle" operation. Data is not yet durably
// persisted at this point.
func (n *ParDo) FinishBundle(_ context.Context) error {

View on GitHub (pinned to 12126d8942)

Solutions

Ensure ProcessElement processes the entire restriction before returning.
If early termination is intentional, checkpoint via the restriction tracker or return the appropriate resumption error instead of nil.
Verify the restriction tracker is being claimed in the element-processing loop and that restriction bounds are correct.

Defensive patterns

Strategy: validation

Validate before calling

// In a custom SDF, verify the restriction is fully processed before returning nil
if !rt.TryClaim(restrictionEnd) { return rt.GetError() } // propagate, never return nil with work left

Prevention

Always drive element processing through the restriction tracker's TryClaim loop
Checkpoint or return a resumption error instead of returning nil with unfinished restriction
Test custom SDFs with restrictions that end mid-element and at boundaries

When it happens

Trigger: A splittable DoFn's ProcessElement returns without processing the entire restriction (the tracker's IsDone/TryClaim path indicates remaining work) and returns nil error; rtErrHelper converts the nil into this error.

Common situations: Custom SDF implementations that stop iterating early without checkpointing or returning a resumption error; restrictions sized incorrectly so the loop ends before the restriction is consumed; forgetting to call the tracker's TryClaim in the processing loop.

Understand the failure class

Background: "This is a bug, please report it": internal invariant violations, unreachable panics, and SNH errors explained — this error's family across 47 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/4e20827a92fda72d. Report an issue: GitHub.