apache/beam · error

observed PCollection has incompatible type: %v

Error message

observed PCollection has incompatible type: %v

What it means

passert.EqualsFloat validates that both the observed and expected PCollections hold non-complex numeric types before comparing with a tolerance. If the expected PCollection's element type is not a valid non-complex number (e.g. string, bool, struct, complex128), the returned reason is formatted into 'observed PCollection has incompatible type: %v'. Comparison is refused rather than performed.

Source

Thrown at sdks/go/pkg/beam/testing/passert/floats.go:62

}

// TryEqualsFloat checks that two PCollections of floats are equal, with each element
// being within a specified threshold of its corresponding element. Both PCollections
// are loaded into memory, sorted, and compared element by element. Returns an error if
// the PCollection types are complex or non-numeric.
func TryEqualsFloat(s beam.Scope, observed, expected beam.PCollection, threshold float64) error {
	errorStrings := []string{}
	observedT := beam.ValidateNonCompositeType(observed)
	if obsErr := validateNonComplexNumber(observedT.Type()); obsErr != nil {
		errorStrings = append(errorStrings, fmt.Sprintf("observed PCollection has incompatible type: %v", obsErr))
	}
	expectedT := beam.ValidateNonCompositeType(expected)
	validateNonComplexNumber(expectedT.Type())
	if expErr := validateNonComplexNumber(expectedT.Type()); expErr != nil {
		errorStrings = append(errorStrings, fmt.Sprintf("expected PCollection has incompatible type: %v", expErr))
	}
	if len(errorStrings) != 0 {
		return errors.New(strings.Join(errorStrings, "\n"))
	}
	s = s.Scope(fmt.Sprintf("passert.EqualsFloat[%v]", threshold))
	beam.ParDo0(s, &thresholdFn{Threshold: threshold}, beam.Impulse(s), beam.SideInput{Input: observed}, beam.SideInput{Input: expected})
	return nil
}

type thresholdFn struct {
	Threshold float64
}

func (f *thresholdFn) ProcessElement(_ []byte, observed, expected func(*beam.T) bool) error {
	var observedValues, expectedValues []float64
	var observedInput, expectedInput beam.T
	for observed(&observedInput) {
		val := toFloat(observedInput)
		observedValues = append(observedValues, val)
	}
	for expected(&expectedInput) {

View on GitHub (pinned to 12126d8942)

Solutions

1. Ensure the observed PCollection's elements are Go numeric types (int*, uint*, float*)
2. Insert a beam.ParDo converting elements to float64 before asserting
3. Use passert.Equals for non-numeric collections instead of EqualsFloat
4. Check the upstream DoFn's emit type matches the intended numeric type

Example fix

// before
passert.EqualsFloat(s, stringCol, 3.14) // incompatible type: string
// after
nums := beam.ParDo(s, &parseFloatFn{}, stringCol) // emits float64
passert.EqualsFloat(s, nums, 3.14)

// verify the observed PCollection element type before asserting
t := observed.Type().Type()
switch t.Kind() {
case reflect.Int, reflect.Int32, reflect.Int64, reflect.Uint32, reflect.Uint64, reflect.Float32, reflect.Float64:
	// ok
default:
	return fmt.Errorf("observed PCollection has incompatible type: %v", t)
}

func isNumericCollection(col beam.PCollection) bool {
	k := col.Type().Type().Kind()
	return k >= reflect.Int && k <= reflect.Float64
}

if err := passert.EqualsFloat(s, observed, expected); err != nil {
	t.Fatalf("type validation failed: %v", err)
}

When it happens

Trigger: Calling EqualsFloat(s, observed, expected) where observed's coder/type is not int/uint/float (or the expected side fails validateNonComplexNumber); TestEqualsFloat_nonNumeric exercises exactly this path.

Common situations: Feeding a PCollection of strings or structs into EqualsFloat; upstream ParDo emitting interface/any values; accidentally passing the wrong PCollection variable.

Understand the failure class

Background: Type mismatch errors: IllegalArgumentException, TypeError and type guards across 150 open-source libraries — this error's family across 150 libraries.

Related errors

• %d missing entries (missing in actual, present in expected)
• values below expected: %v
• values below minimum value %v: %v
• passert.Count(%v) = %v, want %v
• PCollections of different lengths, got %v expected %v

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/8882679dd90a3f9a. Report an issue: GitHub.