apache/beam · error

invalid coder

Error message

invalid coder

What it means

newJSONCoder wraps coder.NewCustomCoder("json", t, jsonEnc, jsonDec). NewCustomCoder validates that the type is actually encodable/decodable by the supplied encoder/decoder functions; if validation fails (e.g. the JSON encoder/decoder function signatures do not type-check against the element type, or the type cannot be used with reflection-based JSON coding), the error is wrapped as "invalid coder".

Source

Thrown at sdks/go/pkg/beam/coder.go:343

// jsonEnc encodes the supplied value in JSON.
func jsonEnc(in T) ([]byte, error) {
	return jsonx.Marshal(in)
}

// jsonDec decodes the supplied JSON into an instance of the supplied type.
func jsonDec(t reflect.Type, in []byte) (T, error) {
	val := reflect.New(t)
	if err := jsonx.Unmarshal(val.Interface(), in); err != nil {
		return nil, err
	}
	return val.Elem().Interface(), nil
}

func newJSONCoder(t reflect.Type) (*coder.CustomCoder, error) {
	c, err := coder.NewCustomCoder("json", t, jsonEnc, jsonDec)
	if err != nil {
		return nil, errors.Wrapf(err, "invalid coder")
	}
	return c, nil
}

// These maps and mutexes are actuated per element, which can be expensive.
var (
	encMu      sync.Mutex
	schemaEncs = map[reflect.Type]func(any, io.Writer) error{}

	decMu      sync.Mutex
	schemaDecs = map[reflect.Type]func(io.Reader) (any, error){}
)

// schemaEnc encodes the supplied value as beam schema.
func schemaEnc(t reflect.Type, in T) ([]byte, error) {
	switch t.Kind() {
	case reflect.Slice, reflect.Array:
		t = t.Elem()

View on GitHub (pinned to 12126d8942)

Solutions

1. Inspect the wrapped cause (errors.Wrapf preserves it) via %v/%+v on the returned error to see why NewCustomCoder rejected the type.
2. Change the PCollection element to a JSON-serializable concrete struct with exported fields.
3. Register a custom coder with your own encoder/decoder functions whose signatures match the element type: coder.NewCustomCoder(name, typ, enc, dec) and pass it explicitly instead of relying on JSON inference.
4. Avoid element types containing channels, funcs, or cyclic references; those cannot be codified.

Example fix

// before
beam.ParDo(s, &Fn{}, col) // element type contains a func field -> invalid coder

// after
type MyElem struct { Value string } // JSON-safe concrete struct
beam.ParDo(s, &Fn{}, col, beam.TypeDefinition{... reflect.TypeOf(MyElem{})})

if _, err := coder.NewCustomCoder("json", reflect.TypeOf(elem{}), jsonEnc, jsonDec); err != nil {
  return fmt.Errorf("element type not JSON-codable: %w", err)
}

func isJSONCodable(t reflect.Type) bool {
  _, err := coder.NewCustomCoder("json", t, jsonEnc, jsonDec)
  return err == nil
}

c, err := newJSONCoder(t)
if err != nil {
  return fmt.Errorf("element type %v rejected by JSON coder: %w", t, err)
}

When it happens

Trigger: inferCoder reaches newJSONCoder(et) for a type it deemed JSON-codable, but coder.NewCustomCoder rejects the (type, encoder, decoder) triple — typically because jsonEnc/jsonDec cannot legally encode/decode that reflect.Type, or the type has no concrete representation (e.g. unexported/recursive structures rejected by the custom-coder validator).

Common situations: Custom coder registration with mismatched encoder/decoder signatures; elements whose types fail reflection-based JSON encode validation (e.g. channels, funcs, or unexported fields in nested structs passed through generic pipelines).

Understand the failure class

Background: "Must be a positive integer", "Invalid value", "Unsupported": the invalid-argument-value error family, when a library rejects the value you pass — this error's family across 35 libraries.

Related errors

• varint too long
• unable to json unmarshal --environment_config: %w
• %v
• unable to rewrite coder %v for state %v for transform %v in
• unknown coder used for ordered list state after re-write id:

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/ea4e3981ee040cf0. Report an issue: GitHub.