apache/beam · error

could not unmarshal stream type coder from %v, stream must b

Error message

could not unmarshal stream type coder from %v, stream must be pair value

What it means

makeCoder encountered a pipelineb.Coder whose URN is the (internal) stream type but whose structure is not the required pair-value shape, so it cannot be converted into a Beam coder. This URN is only valid as a component inside a pair coder; a bare stream-type coder is a protocol violation.

Source

Thrown at sdks/go/pkg/beam/core/runtime/graphx/coder.go:324

		elm, err := b.Coder(components[0])
		if err != nil {
			return nil, err
		}
		w, err := b.WindowCoder(components[1])
		if err != nil {
			return nil, errors.Errorf("could not unmarshal window coder: %w", err)
		}
		t := typex.New(typex.WindowedValueType, elm.T)
		wvc := &coder.Coder{Kind: coder.WindowedValue, T: t, Components: []*coder.Coder{elm}, Window: w}
		if urn == urnWindowedValueCoder {
			return wvc, nil
		}
		wvc.Kind = coder.ParamWindowedValue
		wvc.Window.Payload = string(c.GetSpec().GetPayload())
		return wvc, nil

	case streamType:
		return nil, errors.Errorf("could not unmarshal stream type coder from %v, stream must be pair value", c)

	case "":
		// TODO(herohde) 11/27/2017: we still see CoderRefs from Dataflow. Handle that
		// case here, for now, so that the harness can use this logic.

		payload := c.GetSpec().GetPayload()

		var ref CoderRef
		if err := json.Unmarshal(payload, &ref); err != nil {
			return nil, errors.Wrapf(err, "could not unmarshal CoderRef from %v, failed to decode urn-less coder's payload \"%v\"", c, string(payload))
		}
		c, err := DecodeCoderRef(&ref)
		if err != nil {
			return nil, errors.Wrapf(err, "could not unmarshal CoderRef from %v, failed to decode CoderRef \"%v\"", c, string(payload))
		}
		return c, nil

	case urnIterableCoder, urnStateBackedIterableCoder:

View on GitHub (pinned to 12126d8942)

Solutions

1. Inspect the serialized coder: the stream URN must appear only as a component of a pair coder
2. Re-generate the pipeline with the Go SDK so coders are encoded with supported URNs
3. Upgrade/align SDK versions on both producer and consumer sides
4. If from Dataflow legacy refs, use the modern model pipeline representation

if ref.Urn == streamTypeURN {
    return errors.New("stream type coder must be nested in a pair coder")
}

When it happens

Trigger: Decoding a CoderRef/pipeline protobuf where a coder component uses the stream-type URN directly instead of nested under a pair coder.

Common situations: Graphs produced by non-Go runners or legacy Dataflow CoderRefs leaking through; hand-crafted pipeline JSON; SDK version mismatches producing stale coder encodings.

Understand the failure class

Background: "Invalid ... format", "must be in format X", "does not look like a ..." — invalid argument format errors across CLI tools and libraries — this error's family across 17 libraries.

Related errors

• could not unmarshal CoderRef from %v, failed to decode Coder
• cannot encode a null Integer
• cannot encode a null Long
• cannot encode a null Short
• cannot encode a null BitSet

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/ce94a82b006283ec. Report an issue: GitHub.