apache/beam · error

value %v not encodable with %v

Error message

value %v not encodable with %v

What it means

passert builds an in-memory index of a PCollection by encoding each element with the pipeline's coder, using the encoded bytes as a map key for dedup/counting. If encoding any element fails, index returns 'value %v not encodable with %v'.

Source

Thrown at sdks/go/pkg/beam/testing/passert/passert.go:120

func emitN(val beam.T, n int, emit func(t beam.T)) {
	for i := 0; i < n; i++ {
		emit(val)
	}
}

type indexEntry struct {
	count int
	value beam.T
}

func index(enc beam.ElementEncoder, iter func(*beam.T) bool) (map[string]indexEntry, error) {
	ret := make(map[string]indexEntry)

	var val beam.T
	for iter(&val) {
		var buf bytes.Buffer
		if err := enc.Encode(val, &buf); err != nil {
			return nil, errors.Errorf("value %v not encodable with %v", val, enc)
		}
		encoded := buf.String()

		cur := ret[encoded]
		ret[encoded] = indexEntry{count: cur.count + 1, value: val}
	}
	return ret, nil
}

// True asserts that all elements satisfy the given predicate.
func True(s beam.Scope, col beam.PCollection, fn any) beam.PCollection {
	fail(s, filter.Exclude(s, col, fn), "predicate(%v) = false, want true")
	return col
}

// False asserts that the given predicate does not satisfy any element in the condition.
func False(s beam.Scope, col beam.PCollection, fn any) beam.PCollection {
	fail(s, filter.Include(s, col, fn), "predicate(%v) = true, want false")

View on GitHub (pinned to 12126d8942)

Solutions

1. Ensure the element type has a working registered coder; implement a custom coder if needed.
2. Remove unsupported field types (func, chan, unsafe pointers) from element structs.
3. Test the coder directly on a sample value to reproduce the failure.
4. Convert elements to a simpler encodable type before asserting.

Example fix

// before
type weirdDoFn struct{ cb func(int) } // func field not encodable
// after
type weirdDoFn struct{} // drop non-encodable fields or provide a custom coder

var buf bytes.Buffer
if err := enc.Encode(sampleValue, &buf); err != nil {
	t.Fatalf("element type not encodable, assertion will fail: %v", err)
}

func encodable(v any, enc Coder) bool {
	var buf bytes.Buffer
	return enc.Encode(v, &buf) == nil
}

if err := beamx.Execute(ctx, p); err != nil {
	if strings.Contains(err.Error(), "not encodable with") {
		// register/implement a coder for the failing element type
	}
}

When it happens

Trigger: A ProcessElement-side call to index encounters a value the provided enc.Encode cannot serialize — a type with no registered coder, a func/chan/interface element, or a failing custom coder.

Common situations: Asserting on PCollections containing unsupported types (funcs, channels, structs with non-encodable fields); buggy custom coders; generic interface element types without concrete coders.

Understand the failure class

Background: json.Marshal / "failed to marshal" errors in Go: why "unsupported type" happens and how to fix it — this error's family across 22 libraries.

Related errors

• varint too long
• %d missing entries (missing in actual, present in expected)
• observed PCollection has incompatible type: %v
• values below expected: %v
• values below minimum value %v: %v

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/922353cfa2c64472. Report an issue: GitHub.