apache/beam · error

Duplicate timer family ID %v used by struct fields %v and %v

Error message

Duplicate timer family ID %v used by struct fields %v and %v. Ensure that timer family IDs are unique per DoFn

What it means

Apache Beam Go SDK validates DoFn definitions at graph-construction time. A DoFn may declare multiple Timer struct fields, each with one or more timer family IDs exposed via its Timers() map. When two struct fields expose the same timer family ID, the SDK cannot unambiguously route timer callbacks, so fn.go rejects the DoFn with this error before the pipeline runs.

Source

Thrown at sdks/go/pkg/beam/core/graph/fn.go:1437

func validateTimer(fn *DoFn, numIn mainInputs) error {
	pt, fieldNames := fn.PipelineTimers()

	if _, ok := fn.methods[processElementName].TimerProvider(); ok {
		if numIn == MainSingle {
			err := errors.Errorf("ProcessElement uses a TimerProvider, but is not keyed")
			return errors.SetTopLevelMsgf(err, "ProcessElement uses a TimerProvider, but is not keyed. "+
				"All stateful DoFns must take a key/value pair as an input.")
		}
		if len(pt) == 0 {
			err := errors.New("ProcessElement uses a TimerProvider, but no Timer fields are defined in the DoFn")
			return errors.SetTopLevelMsgf(err, "ProcessElement uses a TimerProvider, but no timer fields are defined in the DoFn"+
				", Ensure that your DoFn exports the Timer fields used to set and clear timers.")
		}
		timerKeys := make(map[string]string)
		for i, t := range pt {
			for timerFamilyID := range t.Timers() {
				if timer, ok := timerKeys[timerFamilyID]; ok {
					err := errors.Errorf("Duplicate timer key %v", timerFamilyID)
					return errors.SetTopLevelMsgf(err, "Duplicate timer family ID %v used by struct fields %v and %v. Ensure that timer family IDs are unique per DoFn", timerFamilyID, timer, fieldNames[i])
				}
				timerKeys[timerFamilyID] = fieldNames[i]
			}
		}
		if err := validateOnTimerFn(fn); err != nil {
			return err
		}
	} else {
		if len(pt) > 0 {
			err := errors.Errorf("ProcessElement doesn't use a TimerProvider, but Timer field is attached to the DoFn: %v", pt)
			return errors.SetTopLevelMsgf(err, "ProcessElement doesn't use a TimerProvider, but Timer field is attached to the DoFn: %v"+
				", Ensure that you are using the TimerProvider to set and clear the timers.", pt)
		}
		if err := validateOnTimerFn(fn); err == nil {
			actualErr := errors.New("OnTimer function is defined for the DoFn but no TimerProvider defined in ProcessElement")
			return errors.SetTopLevelMsgf(actualErr, "OnTimer function is defined for the DoFn but no TimerProvider defined in ProcessElement."+
				"Ensure that timers.Provider is defined in the ProcessElement and OnTimer methods of DoFn.")

View on GitHub (pinned to 12126d8942)

Solutions

1. Assign a unique timer family ID (spec.TimerFamilyID) to each Timer field in the DoFn struct.
2. If both fields were meant to be the same timer, delete the duplicate field and keep a single one.
3. Use distinct Go constants for each timer family instead of a shared one to avoid collisions.
4. Re-run pipeline construction; the error names the two offending struct fields to fix.

Example fix

// before
type MyDoFn struct {
    daily  Timer
    backup Timer
}
// after: give each timer a unique family ID
var dailySpec = timer.NewMainInputTimerSpec("daily", time.Window{})
var backupSpec = timer.NewMainInputTimerSpec("backup", time.Window{})

func checkUniqueTimerFamilies(d interface{}) error {
    seen := map[string]string{}
    v := reflect.ValueOf(d)
    for i := 0; i < v.NumField(); i++ {
        t, ok := v.Field(i).Interface().(Timer)
        if !ok { continue }
        for id := range t.Timers() {
            if prev := seen[id]; prev != "" {
                return fmt.Errorf("duplicate timer family %q on %s and %s", id, prev, v.Type().Field(i).Name)
            }
            seen[id] = v.Type().Field(i).Name
        }
    }
    return nil
}

When it happens

Trigger: Defining a DoFn with two or more Timer-typed fields (e.g. `daily Timer` and `retry Timer`) whose spec.TimerFamilyID values are identical, then calling beam.ParDo/Run so graph validation invokes validateTimerFields in sdks/go/pkg/beam/core/graph/fn.go.

Common situations: Copy-pasting a Timer field and forgetting to change its family ID constant; refactoring timer specs into a shared constant used by two fields; merging two DoFns into one struct and keeping both timer fields.

Understand the failure class

Background: "Invalid value" and "allowed values are" config errors: what your library rejected and how to fix it — this error's family across 41 libraries.

Related errors

• ProcessElement doesn't use a TimerProvider, but Timer field
• first main input parameter must be a value type
• ProcessElement uses a TimerProvider, but no Timer fields are
• bad parameter type for %s: %v
• cannot substitute type %v with %v, already defined as %v

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/d3bffb6d68c952ec. Report an issue: GitHub.