apache/beam · error

not all required SplittableDoFn methods are present. Missing

Error message

not all required SplittableDoFn methods are present. Missing methods: %v

What it means

A SplittableDoFn must implement all required SDF methods (e.g. CreateInitialRestriction, SplitRestriction, RestrictionSize, TruncateRestriction). validateIsSdf, called from AsDoFn, tolerates either zero or all required methods; a partial set (some present, some missing) means an invalid SplittableDoFn and produces this error listing the missing methods.

Source

Thrown at sdks/go/pkg/beam/core/graph/fn.go:870

//   - Not include an RTracker parameter in ProcessElement.
func validateIsSdf(fn *Fn) (bool, error) {
	// Store missing method names so we can output them to the user if validation fails.
	var missing []string
	for _, name := range requiredSdfNames {
		_, ok := fn.methods[name]
		if !ok {
			missing = append(missing, name)
		}
	}

	var isSdf bool
	switch len(missing) {
	case 0: // All SDF methods present.
		isSdf = true
	case len(requiredSdfNames): // No SDF methods.
		isSdf = false
	default: // Anything else means an invalid # of SDF methods.
		err := errors.Errorf("not all required SplittableDoFn methods are present. Missing methods: %v", missing)
		return false, err
	}

	processFn := fn.methods[processElementName]
	if pos, ok := processFn.RTracker(); ok != isSdf {
		if ok {
			err := errors.Errorf("method %v has sdf.RTracker as param %v, expected none",
				processElementName, pos)
			return false, errors.SetTopLevelMsgf(err, "Method %v has an sdf.RTracker parameter at index %v, "+
				"but is not part of a splittable DoFn. sdf.RTracker is invalid in %v in non-splittable DoFns.",
				processElementName, pos, processElementName)
		}
		pos, _, _ = processFn.Inputs()
		err := errors.Errorf("method %v missing sdf.RTracker, expected one at index %v",
			processElementName, pos)
		return false, errors.SetTopLevelMsgf(err, "Method %v is missing an sdf.RTracker "+
			"parameter despite being part of a splittable DoFn. %v in splittable DoFns requires an "+
			"sdf.RTracker parameter before main inputs (in this case, at index %v).",

View on GitHub (pinned to 12126d8942)

Solutions

  1. Implement every missing method listed in the error message (all requiredSdfNames).
  2. If splitting is not intended, remove all SDF methods so the DoFn is a plain DoFn.
  3. Verify the sdf.RTracker parameter in ProcessElement matches a fully implemented SDF.

Example fix

// before
func (fn *myFn) CreateInitialRestriction(rangeSize int) offsetrange.Restriction { ... }
// missing SplitRestriction / RestrictionSize

// after
func (fn *myFn) CreateInitialRestriction(rangeSize int) offsetrange.Restriction { ... }
func (fn *myFn) SplitRestriction(rs offsetrange.Restriction) []offsetrange.Restriction { ... }
func (fn *myFn) RestrictionSize(rs offsetrange.Restriction) float64 { ... }
Defensive patterns

Strategy: validation

Validate before calling

// before registering an SDF, assert all required methods exist:
required := []string{"CreateInitialRestriction", "SplitRestriction", "RestrictionSize"}
for _, m := range required {
	if _, ok := reflect.TypeOf(fn).MethodByName(m); !ok { return fmt.Errorf("missing %s", m) }
}

Type guard

func isCompleteSdf(fn interface{}) bool { t := reflect.TypeOf(fn); for _, m := range requiredSdfNames { if _, ok := t.MethodByName(m); !ok { return false } }; return true }

Try / catch

if !isCompleteSdf(&myFn{}) { /* implement all SDF methods or drop RTracker */ }
if err := beam.ParDo(s, &myFn{}, in); err != nil { log.Fatalf("SDF incomplete: %v", err) }

Prevention

When it happens

Trigger: Implementing only some SDF methods on a DoFn (e.g. CreateInitialRestriction but not RestrictionSize), then registering it with beam.ParDo during graph construction; also occurs if a required method was deleted or renamed.

Common situations: Partially migrating a classic DoFn to an SDF; a rename/refactor removing one required method; copying an SDF example but omitting a method the author thought optional.

Understand the failure class

Background: "missing required argument" and "the following required arguments were not provided": what required-argument errors mean and how to fix them — this error's family across 20 libraries.

Related errors


AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/e92cb56285e8ac41. Report an issue: GitHub.