apache/beam · error

unable to rewrite coder %v for state %v for transform %v in

Error message

unable to rewrite coder %v for state %v for transform %v in stage %v:%w

What it means

When prism rewrites state coder IDs in a ParDoPayload so coders exist in the bundle descriptor, lpUnknownCoders may fail; this error wraps that failure with the coder ID, state ID, transform, and stage. The original cause is preserved via %w.

Source

Thrown at sdks/go/pkg/beam/runners/prism/internal/stage.go:459

		if t.GetSpec().GetUrn() != urns.TransformParDo {
			continue
		}

		pardo := &pipepb.ParDoPayload{}
		if err := (proto.UnmarshalOptions{}).Unmarshal(t.GetSpec().GetPayload(), pardo); err != nil {
			return fmt.Errorf("unable to decode ParDoPayload for %v in stage %v", tid, stg.ID)
		}

		// We need to ensure the coders can be handled by prism, and are available in the bundle descriptor.
		// So we rewrite the transform's Payload with updated coder ids here.
		var rewrite bool
		var rewriteErr error
		for stateID, s := range pardo.GetStateSpecs() {
			rewrite = true
			rewriteCoder := func(cid *string) {
				newCid, err := lpUnknownCoders(*cid, coders, comps.GetCoders())
				if err != nil {
					rewriteErr = fmt.Errorf("unable to rewrite coder %v for state %v for transform %v in stage %v:%w", *cid, stateID, tid, stg.ID, err)
					return
				}
				*cid = newCid
			}
			switch s := s.GetSpec().(type) {
			case *pipepb.StateSpec_BagSpec:
				rewriteCoder(&s.BagSpec.ElementCoderId)
			case *pipepb.StateSpec_SetSpec:
				rewriteCoder(&s.SetSpec.ElementCoderId)
			case *pipepb.StateSpec_OrderedListSpec:
				rewriteCoder(&s.OrderedListSpec.ElementCoderId)
				// Add the length determination helper for OrderedList state values.
				if stg.stateTypeLen == nil {
					stg.stateTypeLen = map[engine.LinkID]func([]byte) int{}
				}
				linkID := engine.LinkID{
					Transform: tid,
					Local:     stateID,

View on GitHub (pinned to 12126d8942)

Solutions

1. Read the wrapped cause (%w) for why the coder rewrite failed
2. Ensure all state coders are standard supported coders (bytes, varint, length-prefix)
3. Check the pipeline components section includes the referenced coder definitions
4. Avoid custom coders for state in pipelines targeting prism, or upgrade Beam

for cid := range pardo.GetStateSpecs() {
    if comps.GetCoders()[cid] == nil {
        return fmt.Errorf("state coder %v missing from components", cid)
    }
}

if err := buildDescriptor(...); err != nil {
    var wrappedErr error
    if errors.As(err, &wrappedErr) && strings.Contains(err.Error(), "unable to rewrite coder") {
        // inspect wrapped cause via errors.Unwrap
    }
    return err
}

When it happens

Trigger: A state spec (bag, map, set, etc.) references a coder ID that lpUnknownCoders cannot find or rewrite in comps.GetCoders() — typically a dangling or unsupported coder reference in the pipeline components.

Common situations: Stateful DoFns from SDKs whose generated coders prism can't handle; cross-language stateful transforms with unknown coders; version skew altering coder IDs.

Related errors

• unknown coder used for ordered list state after re-write id:
• varint too long
• opening a reader %v on a closed channel. Original error: %w
• %v
• invalid coder

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/81b06307bdc35016. Report an issue: GitHub.