apache/beam · error

instruction no longer processing

Error message

instruction %v no longer processing

What it means

ScopedStateReader.openReader in the Beam Go harness refuses to open a new side-input/user-state reader when the scoped state reader has already been closed for the current bundle instruction. Once an instruction's processing finishes, its state channels are torn down; opening a reader afterwards would target a dead instruction. The library throws this to prevent reads against state that will never be answered.

Solutions

1. Ensure all state/side-input reads happen inside the ProcessElement/StartBundle lifecycle, before Close() is called on the state reader.
2. Fix goroutine leaks or deferred callbacks in DoFns that touch user state after the bundle ends.
3. Check bundle/instruction lifecycle handling in custom exec code; do not cache ScopedStateReader across bundles.
4. If caused by a Beam runtime bug (e.g. state access in teardown), report with the pipeline and upgrade to the latest Beam Go SDK version.

Example fix

// before (goroutine outlives bundle)
go func() { it, _ := ctx.sideInput(ctx) }()
// after: perform side-input reads synchronously inside ProcessElement
it := s.side.Read(ctx, s.sideInput)

if s.IsClosed() { return fmt.Errorf("skip: state scope closed for instruction") }

iter, err := reader.OpenIterableSideInput(ctx, id, w)
if err != nil {
    if strings.Contains(err.Error(), "no longer processing") {
        return nil // bundle ended; abort read gracefully
    }
    return err
}

When it happens

Trigger: Calling OpenIterableSideInput, OpenMultiMapSideInput, OpenIterable, or any Open*UserStateReader after the ScopedStateReader was closed (bundle ended or Close() invoked), typically via a DoFn continuing to touch user state or side inputs after the bundle context finished.

Common situations: DoFn goroutines outliving the bundle (FinishBundle or process cleanup racing with state access), reusing exec units across bundles after close, async/callback code firing after the harness closed the instruction's state scope.

Understand the failure class

Background: "Invalid state transition" errors: "status must be X, actually Y", "already rejected/charging/uninstalled", "cannot ... while running" — what they mean when a library rejects your call — this error's family across 31 libraries.

Related errors

• invalid status for combine convert
• invalid status for combine extract
• invalid status for combine merge
• invalid status for combine
• invalid status for precombine

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/4345df5c18344391. Report an issue: GitHub.

Appendix: source

Thrown at sdks/go/pkg/beam/core/runtime/harness/statemgr.go:188

	})
	return wr, err
}

// GetSideInputCache returns a pointer to the SideInputCache being used by the SDK harness.
func (s *ScopedStateReader) GetSideInputCache() exec.SideCache {
	return s.cache
}

func (s *ScopedStateReader) openReader(ctx context.Context, id exec.StreamID, readerFn func(*StateChannel) *stateKeyReader) (*stateKeyReader, error) {
	ch, err := s.open(ctx, id.Port)
	if err != nil {
		return nil, err
	}

	s.mu.Lock()
	if s.closed {
		s.mu.Unlock()
		return nil, errors.Errorf("instruction %v no longer processing", s.instID)
	}
	ret := readerFn(ch)
	s.mu.Unlock()
	return ret, nil
}

func (s *ScopedStateReader) openWriter(ctx context.Context, id exec.StreamID, writerFn func(*StateChannel) *stateKeyWriter) (*stateKeyWriter, error) {
	ch, err := s.open(ctx, id.Port)
	if err != nil {
		return nil, err
	}

	s.mu.Lock()
	if s.closed {
		s.mu.Unlock()
		return nil, errors.Errorf("instruction %v no longer processing", s.instID)
	}
	ret := writerFn(ch)

View on GitHub (pinned to 12126d8942)