apache/beam · error

failed to marshal iterable coder %v

Error message

failed to marshal iterable coder %v

What it means

Produced by CoderMarshaller.Add when marshalling a coder.Iterable: it recursively marshals the iterable's component coders via AddMulti, and wraps any failure with "failed to marshal iterable coder". The SDK throws it because an iterable coder can only be serialized if all of its component coders serialize successfully.

Source

Thrown at sdks/go/pkg/beam/core/runtime/graphx/coder.go:577

		comp := []string{}
		ids, err := b.AddMulti(c.Components)
		if err != nil {
			return "", errors.SetTopLevelMsgf(err, "failed to marshal timer coder %v", c)
		}
		comp = append(comp, ids...)

		id, err := b.AddWindowCoder(c.Window)
		if err != nil {
			return "", errors.Wrapf(err, "failed to marshal window coder %v", c)
		}
		comp = append(comp, id)

		return b.internBuiltInCoder(urnTimerCoder, comp...), nil

	case coder.Iterable:
		comp, err := b.AddMulti(c.Components)
		if err != nil {
			return "", errors.Wrapf(err, "failed to marshal iterable coder %v", c)
		}
		return b.internBuiltInCoder(urnIterableCoder, comp...), nil

	default:
		err := errors.Errorf("unexpected coder kind: %v", c.Kind)
		return "", errors.WithContextf(err, "failed to marshal coder %v", c)
	}
}

// AddMulti adds the given coders to the set and returns their ids. Idempotent.
func (b *CoderMarshaller) AddMulti(list []*coder.Coder) ([]string, error) {
	var ids []string
	for _, c := range list {
		if id, err := b.Add(c); err != nil {
			return nil, errors.Wrapf(err, "failed to marshal the coder %v.", c)
		} else {
			ids = append(ids, id)
		}

View on GitHub (pinned to 12126d8942)

Solutions

1. Read the wrapped cause to find which nested component coder failed.
2. Replace unsupported element types in the iterable with types that have registered encoders.
3. Register a custom coder via beam.Encoder/beam.Decoder registration for unsupported element types.
4. Validate the whole pipeline coder with CoderMarshaller.AddMulti before submission to isolate the failing component.
5. Upgrade to a Beam version that supports the element kind in question.

Example fix

// before
iterCoder := coder.NewI(coder.NewCustomCoder(unknownKind))
// after
iterCoder := coder.NewI(coder.NewBytes()) // use a coder kind the marshaller supports

for _, comp := range c.Components {
    if comp == nil {
        return errors.New("iterable has nil component coder")
    }
}

func isKnownKind(c *coder.Coder) bool {
    switch c.Kind {
    case coder.Custom, coder.KV, coder.Window, coder.Timer, coder.Iterable, coder.Bool, coder.Bytes:
        return true
    }
    return false
}

ids, err := b.AddMulti(c.Components)
if err != nil {
    return "", fmt.Errorf("iterable coder %v component failed: %w", c, err)
}

When it happens

Trigger: Add(c) where c.Kind == coder.Iterable and AddMulti(c.Components) fails — e.g. a nested component has an unsupported kind or a failing window coder.

Common situations: Pipelines with PCollections of lists/iterables whose element type triggers a nested marshalling error; custom element coders that hit the 'unexpected coder kind' default path; malformed coders built manually.

Understand the failure class

Background: json.Marshal / "failed to marshal" errors in Go: why "unsupported type" happens and how to fix it — this error's family across 22 libraries.

Related errors

• empty type
• invalid float encoding for: %v
• unable to decode array with iterable marker %v
• could not unmarshal stream type coder from %v, stream must b
• could not unmarshal CoderRef from %v, failed to decode Coder

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/071c724cdc7bd952. Report an issue: GitHub.