apache/beam · critical

nothing in progress and no refreshes with non zero pending…

Error message

nothing in progress and no refreshes with non zero pending elements: %v
%v

What it means

Prism's element manager detects quiescence of a running pipeline. If there are no bundles in progress, no refreshes, yet elements remain pending, the runner is in an impossible state — the job would hang forever. Prism fails fast with this error and dumps all stages to help debug, because this indicates a bug in prism's scheduling.

Solutions

1. File a bug with the Beam project including the DumpStages output embedded in the error message.
2. Work around by running the same pipeline with the direct or Flink runner to unblock the job.
3. Check whether a recent Beam version fixed the issue and upgrade sdk/runner.
4. Reduce the pipeline to a minimal reproducer to attach to the bug report.

if err := beam.Run(ctx, prismRunner, p); err != nil && strings.Contains(err.Error(), "nothing in progress and no refreshes") {
    // rerun with a different runner and file a Beam bug with the DumpStages output
}

When it happens

Trigger: checkForQuiescence runs (invoked from an anonymous caller in the engine loop) and finds pending element count > 0 while no bundles are executing and no refreshes are scheduled.

Common situations: Hitting a prism runner bug while executing a pipeline; often related to specific transform/data patterns that starve a stage of schedulable bundles. Always a runner-internal bug, not user config.

Understand the failure class

Background: "This is a bug, please report it": internal invariant violations, unreachable panics, and SNH errors explained — this error's family across 47 libraries.

Related errors

• couldn't decode characteristic for variant
• error decoding append bag user state window key
• error decoding residual header:
• error decoding watermarks
• error re-encoding characteristic for variant

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/aceebaf5360abd0b. Report an issue: GitHub.

Appendix: source

Thrown at sdks/go/pkg/beam/runners/prism/internal/engine/elementmanager.go:634

			// It's impossible to fully control processing time SDK side handling for processing time
			// Runner side, so we specialize refresh handling here to avoid spuriously getting stuck.
			em.changedStages.insert(em.testStreamHandler.ID)
			return nil
		}
		// If there are no changed stages due to a test stream event
		// then there's no mechanism to make progress, so it's time to fast fail.
	}

	v := em.livePending.Load()
	if v == 0 {
		// Since there are no further pending elements, the job will be terminating successfully.
		return nil
	}
	// The job is officially stuck. Fail fast and produce debugging information.
	// Jobs must never get stuck so this indicates a bug in prism to be investigated.

	slog.Debug("Bundles: nothing in progress and no refreshes", slog.Int64("pendingElementCount", v))
	return errors.Errorf("nothing in progress and no refreshes with non zero pending elements: %v\n%v", v, em.DumpStages())
}

// InputForBundle returns pre-allocated data for the given bundle, encoding the elements using
// the PCollection's coders.
func (em *ElementManager) InputForBundle(rb RunBundle, info PColInfo) [][]byte {
	ss := em.stages[rb.StageID]
	ss.mu.Lock()
	defer ss.mu.Unlock()
	es := ss.inprogress[rb.BundleID]
	return es.ToData(info)
}

// DataAndTimerInputForBundle returns pre-allocated data for the given bundle and the estimated number of data elements.
// Elements are encoded with the PCollection's coders.
func (em *ElementManager) DataAndTimerInputForBundle(rb RunBundle, info PColInfo) ([]*Block, int) {
	ss := em.stages[rb.StageID]
	ss.mu.Lock()
	defer ss.mu.Unlock()

View on GitHub (pinned to 12126d8942)