apache/beam · error

marshalling of failed

Error message

marshalling of %v failed

What it means

After the per-element type check, createList encodes each value with the element encoder. This error wraps a failure of ElementEncoder.Encode, meaning a value of the correct type could not be serialized into the pipeline's coder representation.

Solutions

1. Register a custom ElementEncoder/coder for the value's type via graph coders or use an encodable type (primitives, protos, registered types)
2. Simplify the element type to one with built-in encoding support
3. Check the wrapped inner error to see which coder failed and fix its Encode implementation

Example fix

// before
pc, err := beam.TryCreate(s, []MyComplexStruct{...}) // no coder registered
// after
beam.RegisterCoder(reflect.TypeOf((*MyComplexStruct)(nil)).Elem(), encMyStruct, decMyStruct)
pc, err := beam.TryCreate(s, []MyComplexStruct{...})

enc := beam.NewElementEncoder(t); var b bytes.Buffer; for _, v := range values { if err := enc.Encode(v, &b); err != nil { return err } }

pc, err := beam.TryCreate(s, values)
if err != nil {
    if strings.Contains(err.Error(), "marshalling of") { /* register a coder for the element type or simplify the type */ }
    return PCollection{}, err
}

When it happens

Trigger: Encoding a value whose type has no registered element encoder (unsupported custom struct without a coder), a custom coder's Encode method returning an error, or an underlying io.Writer failure.

Common situations: beam.Create on structs with unencodable fields or without a registered custom coder, encoding via a coder registered for a different type, or graph-construction-time coder registration bugs.

Related errors

• encoding , expected
• bad combinefn
• bad decoding function
• bad encoding function
• bad function

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/edef0ef72dd4e39d. Report an issue: GitHub.

Appendix: source

Thrown at sdks/go/pkg/beam/create.go:98

	return createList(s, ret, t)
}

func addCreateCtx(err error, s Scope) error {
	return errors.WithContextf(err, "inserting Create in scope %s", s)
}

func createList(s Scope, values []any, t reflect.Type) (PCollection, error) {
	fn := &createFn{Type: EncodedType{T: t}}
	enc := NewElementEncoder(t)

	for i, value := range values {
		if other := reflect.ValueOf(value).Type(); other != t {
			err := errors.Errorf("value %v at index %v has type %v, want %v", value, i, other, t)
			return PCollection{}, addCreateCtx(err, s)
		}
		var buf bytes.Buffer
		if err := enc.Encode(value, &buf); err != nil {
			err = errors.Wrapf(err, "marshalling of %v failed", value)
			return PCollection{}, addCreateCtx(err, s)
		}
		fn.Values = append(fn.Values, buf.Bytes())
	}

	imp := Impulse(s)

	ret, err := TryParDo(s, fn, imp, TypeDefinition{Var: TType, T: t})
	if err != nil || len(ret) != 1 {
		panic(addCreateCtx(errors.WithContext(err, "internal error"), s))
	}
	return ret[0], nil
}

// TODO(herohde) 6/26/2017: make 'create' a SDF once supported. See BEAM-2421.

type createFn struct {
	Values [][]byte    `json:"values"`

View on GitHub (pinned to 12126d8942)