{"record":{"id":"55e7ca0ddfbb5a0e","repo":"apache/beam","slug":"generating-bundle-for-stage-v-at-watermark-v-panicked-v","errorCode":null,"errorMessage":"generating bundle for stage %v at watermark %v panicked\n%v","messagePattern":"generating bundle for stage (.+?) at watermark (.+?) panicked\n(.+?)","errorType":"panic","errorClass":null,"httpStatus":null,"severity":"critical","filePath":"sdks/go/pkg/beam/runners/prism/internal/engine/elementmanager.go","lineNumber":1877,"sourceCode":"\tss.mu.Lock()\n\tdefer ss.mu.Unlock()\n\treturn ss.output\n}\n\n// TODO: Move to better place for configuration\nvar (\n\tOneKeyPerBundle  bool // OneKeyPerBundle sets if a bundle is restricted to a single key.\n\tOneElementPerKey bool // OneElementPerKey sets if a key in a bundle is restricted to one element.\n)\n\n// startBundle initializes a bundle with elements if possible.\n// A bundle only starts if there are elements at all, and if it's\n// an aggregation stage, if the windowing stratgy allows it.\n// Returns a non-zero adjustment to the pending elements count if the stage is accumulating.\nfunc (ss *stageState) startEventTimeBundle(watermark mtime.Time, genBundID func() string) (string, bool, bool, int) {\n\tdefer func() {\n\t\tif e := recover(); e != nil {\n\t\t\tpanic(fmt.Sprintf(\"generating bundle for stage %v at watermark %v panicked\\n%v\", ss.ID, watermark, e))\n\t\t}\n\t}()\n\tss.mu.Lock()\n\tdefer ss.mu.Unlock()\n\ttoProcess, minTs, newKeys, holdsInBundle, panesInBundle, stillSchedulable, accumulatingPendingAdjustment := ss.kind.buildEventTimeBundle(ss, watermark)\n\n\tif len(toProcess) == 0 {\n\t\t// If we have nothing, there's nothing to progress.\n\t\treturn \"\", false, stillSchedulable, accumulatingPendingAdjustment\n\t}\n\n\tbundID := ss.makeInProgressBundle(genBundID, toProcess, minTs, newKeys, holdsInBundle, panesInBundle)\n\tslog.Debug(\"started an event time bundle\", \"stageID\", ss.ID, \"bundleID\", bundID, \"bundleSize\", len(toProcess), \"upstreamWatermark\", watermark)\n\n\treturn bundID, true, stillSchedulable, accumulatingPendingAdjustment\n}\n\n// buildEventTimeBundle for ordinary stages processes all pending elements.","sourceCodeStart":1859,"sourceCodeEnd":1895,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/go/pkg/beam/runners/prism/internal/engine/elementmanager.go#L1859-L1895","documentation":"startEventTimeBundle wraps bundle creation for event-time-triggered aggregation stages in a recover, and re-panics with the stage ID and watermark attached. This is not an independent failure but a context wrapper: any panic inside buildEventTimeBundle (e.g. the zero-length key panics or hold-tracker panics) surfaces as this message. The root cause is the wrapped inner panic text appended after \\n%v.","triggerScenarios":"Any panic inside ss.kind.buildEventTimeBundle — most commonly zero-length keys in pending elements, or hold-count corruption — while starting an event-time bundle at the given watermark.","commonSituations":"Streaming pipelines with event-time triggers/aggregation where upstream stages emitted malformed elements (empty keys) or where watermark hold bookkeeping got out of sync.","solutions":["Read the wrapped inner panic after \\n in the message — fix that root cause first (often 'zero length key' or hold tracker errors)","Inspect pending elements for the named stage for empty keys before the aggregation stage","Verify trigger/watermark configuration for the stage; misaligned triggering can expose latent data issues","If reproducible on stock pipelines, capture the full panic and report it to the Beam project"],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"// Parse the wrapped inner panic to find the root cause\nparts := strings.SplitN(msg, \"\\n\", 2)\nif len(parts) == 2 {\n    rootCause := parts[1]\n    log.Printf(\"bundle panic root cause: %s\", rootCause)\n}","preventionTips":["Always read the wrapped panic text after the newline for the real cause","Fix the underlying zero-length-key / hold errors, not this wrapper","Keep trigger and watermark configurations conservative in tests"],"tags":["go","beam","prism","panic","watermark"],"backgroundTag":"internal-invariant-violation","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}