apache/beam · error

failed to marshal the coder

Error message

failed to marshal the coder %v.

What it means

Produced by CoderMarshaller.AddMulti, which iterates a list of coders and calls Add for each. If any individual coder fails to marshal, the error is wrapped as 'failed to marshal the coder %v.' identifying the specific failing coder. It exists so batch marshalling (e.g. for composite coders) names the offending element.

Solutions

1. From the message, identify the exact failing coder %v and inspect its Kind and components.
2. Fix or replace that coder with a supported kind.
3. Marshalling the list element-by-element yourself to pinpoint failures early.
4. Ensure every PCollection's type has a default coder Beam can build (registered custom encoders if needed).
5. Align Beam versions across SDKs used in the pipeline.

Example fix

// before
ids, err := b.AddMulti([]*coder.Coder{good, badUnknownCoder})
// after
ids, err := b.AddMulti([]*coder.Coder{good, coder.NewBytes()}) // replace unsupported coder

for i, c := range list {
    if c == nil {
        return fmt.Errorf("coder at index %d is nil", i)
    }
}

func allMarshalable(list []*coder.Coder) bool {
    for _, c := range list {
        if c == nil || c.Kind == 0 {
            return false
        }
    }
    return true
}

ids, err := b.AddMulti(list)
if err != nil {
    return nil, fmt.Errorf("AddMulti: %w", err) // message names the failing coder
}

When it happens

Trigger: Calling AddMulti with a slice containing any coder that Add rejects (unsupported kind, failing window coder, failing nested components); also reached whenever Add is called and any underlying marshalling step fails.

Common situations: Composite (KV/Iterable) coders with one bad component; MarshalCoders at pipeline serialization time when some PCollection has an unsupported coder; cross-language expansion producing coders unknown to Go.

Understand the failure class

Background: json.Marshal / "failed to marshal" errors in Go: why "unsupported type" happens and how to fix it — this error's family across 22 libraries.

Related errors

• bad base type
• bad channel direction
• bad element type
• bad field type
• bad parameter type

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/8d1e22c32f261182. Report an issue: GitHub.

Appendix: source

Thrown at sdks/go/pkg/beam/core/runtime/graphx/coder.go:592

	case coder.Iterable:
		comp, err := b.AddMulti(c.Components)
		if err != nil {
			return "", errors.Wrapf(err, "failed to marshal iterable coder %v", c)
		}
		return b.internBuiltInCoder(urnIterableCoder, comp...), nil

	default:
		err := errors.Errorf("unexpected coder kind: %v", c.Kind)
		return "", errors.WithContextf(err, "failed to marshal coder %v", c)
	}
}

// AddMulti adds the given coders to the set and returns their ids. Idempotent.
func (b *CoderMarshaller) AddMulti(list []*coder.Coder) ([]string, error) {
	var ids []string
	for _, c := range list {
		if id, err := b.Add(c); err != nil {
			return nil, errors.Wrapf(err, "failed to marshal the coder %v.", c)
		} else {
			ids = append(ids, id)
		}
	}
	return ids, nil
}

// AddWindowCoder adds a window coder.
func (b *CoderMarshaller) AddWindowCoder(w *coder.WindowCoder) (string, error) {
	switch w.Kind {
	case coder.GlobalWindow:
		return b.internBuiltInCoder(urnGlobalWindow), nil
	case coder.IntervalWindow:
		return b.internBuiltInCoder(urnIntervalWindow), nil
	default:
		err := errors.Errorf("window coder with unexpected type %v", w.Kind)
		return "", errors.WithContextf(err, "failed to unmarshal window coder %v", w)
	}

View on GitHub (pinned to 12126d8942)