{"record":{"id":"502fc897c50152bf","repo":"apache/beam","slug":"failed-to-split-splittable-unit-was-nil","errorCode":null,"errorMessage":"failed to split: splittable unit was nil","messagePattern":"failed to split: splittable unit was nil","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"sdks/go/pkg/beam/core/runtime/exec/datasource.go","lineNumber":577,"sourceCode":"\t\tfrac = 0.0\n\t}\n\n\tn.mu.Lock()\n\tdefer n.mu.Unlock()\n\n\tvar currProg float64 // Current element progress.\n\tvar su SplittableUnit\n\tif n.index < 0 { // Progress is at the end of the non-existant -1st element.\n\t\tcurrProg = 1.0\n\t} else if n.su == nil { // If this isn't sub-element splittable, estimate some progress.\n\t\tcurrProg = 0.5\n\t} else { // If this is sub-element splittable, get progress of the current element.\n\n\t\tselect {\n\t\tcase su = <-n.su:\n\t\t\t// If an element is processing, we'll get a splittable unit.\n\t\t\tif su == nil {\n\t\t\t\treturn SplitResult{}, fmt.Errorf(\"failed to split: splittable unit was nil\")\n\t\t\t}\n\t\t\tdefer func() {\n\t\t\t\tn.su <- su\n\t\t\t}()\n\t\t\tcurrProg = su.GetProgress()\n\t\tcase <-time.After(500 * time.Millisecond):\n\t\t\t// Otherwise, the current element hasn't started processing yet\n\t\t\t// or has already finished. By adding a short timeout, we avoid\n\t\t\t// the first possibility, and can assume progress is at max.\n\t\t\tcurrProg = 1.0\n\t\t}\n\t}\n\t// Size to split within is the minimum of bufSize or splitIdx so we avoid\n\t// including elements we already know won't be processed.\n\tif bufSize <= 0 || n.splitIdx < bufSize {\n\t\tbufSize = n.splitIdx\n\t}\n\ts, fr, err := splitHelper(n.index, bufSize, currProg, splits, frac, su != nil)","sourceCodeStart":559,"sourceCodeEnd":595,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/go/pkg/beam/core/runtime/exec/datasource.go#L559-L595","documentation":"When a DataSource processes sub-element-splittable elements, the current splittable unit is passed over the n.su channel. If a splittable unit is received but is nil, Beam cannot determine split progress and fails. This indicates the pipeline handed a nil splittable unit where a live one was expected.","triggerScenarios":"Calling Split on a source whose current element is sub-element-splittable, but the splittable unit channel yields nil (element finished processing concurrently, or the source produced a nil su).","commonSituations":"Race between element completion and split request; Splittable DoFn implementations that emit nil restrictions; splitting during the boundary instant when an element just finished.","solutions":["Retry the split; this can be a transient race between element completion and the split request.","Check the Splittable DoFn's restriction/size functions never produce nil or zero-size restrictions.","Verify the custom splitter's CreateRestricter/RestrictionTracker implementations are correct for the element type.","Upgrade the Beam Go SDK; several split-progress races have been fixed over time."],"exampleFix":"// before\nsu = <-n.su // assumes non-nil\n// after (caller-side resilience)\nres, err := source.Split(ctx, splits, frac, bufSize)\nif err != nil && strings.Contains(err.Error(), \"splittable unit was nil\") {\n    time.Sleep(50 * time.Millisecond)\n    res, err = source.Split(ctx, splits, frac, bufSize) // retry after element completes\n}","handlingStrategy":"retry","validationCode":"select {\ncase su := <-n.su:\n    if su == nil { return errors.New(\"no active splittable element; retry later\") }\ndefault:\n}","typeGuard":"func hasActiveSplittableUnit(ch chan *SplittableUnit) bool { select { case su := <-ch: return su != nil; default: return false } }","tryCatchPattern":"res, err := source.Split(ctx, splits, frac, bufSize)\nif err != nil && strings.Contains(err.Error(), \"splittable unit was nil\") {\n    time.Sleep(100 * time.Millisecond)\n    res, err = source.Split(ctx, splits, frac, bufSize)\n}","preventionTips":["Ensure SDF restriction trackers never return nil units","Treat nil-su splits as transient and retry","Keep the Beam Go SDK up to date for split race fixes"],"tags":["beam-go","split","splittable-dofn","race-condition"],"backgroundTag":"internal-invariant-violation","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}