apache/beam · error

broken stream: %v

Error message

broken stream: %v

What it means

This generated single-value iterator reads elements from a runtime stream (FullValue stream). io.EOF is the normal end-of-stream signal; any other read error means the data channel broke mid-stream, so the generated code panics because iterating cannot continue meaningfully. It surfaces as a crash during element iteration in an optimized pipeline step.

Source

Thrown at sdks/go/pkg/beam/core/runtime/exec/optimized/inputs.tmpl:75

	return v.fn
}

func (v *iterNative) Reset() error {
	if err := v.cur.Close(); err != nil {
		return err
	}
	v.cur = nil
	return nil
}

{{range $x := .X}}
func (v *iterNative) read{{$x.Name}}(val *{{$x.Type}}) bool {
	elm, err := v.cur.Read()
	if err != nil {
		if err == io.EOF {
			return false
		}
		panic(fmt.Sprintf("broken stream: %v", err))
	}
    *val = elm.Elm.({{$x.Type}})
    return true
}

func iterMaker{{$x.Name}}(s exec.ReStream) exec.ReusableInput {
	ret := &iterNative{s: s}
	ret.fn = ret.read{{$x.Name}}
	return ret
}

{{range $y := .Y}}
func (v *iterNative) read{{$x.Name}}{{$y.Name}}(key *{{$x.Type}}, value *{{$y.Type}}) bool {
	elm, err := v.cur.Read()
	if err != nil {
		if err == io.EOF {
			return false
		}

View on GitHub (pinned to 12126d8942)

Solutions

1. Check bundle/worker logs for the root cause of the stream failure (network errors, OOM, source failure)
2. Retry the pipeline/job — broken streams are often transient infrastructure failures
3. If reproducible, inspect the upstream source/transform producing the stream for errors, and upgrade Beam to get improved stream error propagation

defer func() {
    if r := recover(); r != nil {
        if strings.HasPrefix(fmt.Sprint(r), "broken stream:") {
            log.Printf("stream broke during iteration: %v — retry bundle", r)
        }
        panic(r)
    }
}()

When it happens

Trigger: A side-input or GBK-style ReStream's underlying reader returns a non-EOF error (broken gRPC data channel, source failure, deserialization error) while readX is pulling the next element.

Common situations: Bundle failures mid-stream — e.g., the runner kills the data connection, network interruption to a remote runner, or an upstream source error corrupting the stream.

Related errors

• broken stream
• broken stream: %v
• broken stream: %v
• trigger.AfterCount(%v) must be a positive integer
• can't apply processing delay of less than a millisecond. Got

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/605a3ae1e5c0a495. Report an issue: GitHub.