apache/beam · error

BytesValue unmarshal failed

Error message

BytesValue unmarshal failed

What it means

Wrap returned by UnpackBytes when proto.Unmarshal of the Any's value into a wrappers.BytesValue fails. The TypeUrl check has already passed, so the payload bytes exist but are not a validly encoded BytesValue proto — corrupt or version-incompatible packed data. Callers such as UnpackProto and UnpackBase64Proto surface this directly.

Solutions

1. Verify the payload bytes are a complete, unmodified protobuf serialization of BytesValue
2. Re-pack the data with PackBytes on the producer side instead of manually assembling the Any
3. Log/inspect data.Value length and bytes to detect truncation or corruption in transit

Example fix

// before
packed, _ := ptypes.MarshalAny(&wrapperspb.BytesValue{Value: data}) // manual assembly risk
// after
packed, err := protox.PackBytes(data)
if err != nil { return err }

if len(data.GetValue()) == 0 { return errors.New("Any payload is empty; not a valid BytesValue") }

v, err := protox.UnpackBytes(anyMsg)
if err != nil {
    return fmt.Errorf("unpacking %T payload: %w", anyMsg, err)
}

When it happens

Trigger: proto.Unmarshal in UnpackBytes returns an error because data.Value is corrupted, truncated, empty, or was serialized by an incompatible proto version.

Common situations: Hand-crafted or corrupted Any payloads, binary data mangled by a transport that is not 8-bit clean, or mixing proto2/proto3 wire formats across pipeline stages.

Understand the failure class

Background: "cannot parse invalid wire-format data", "cannot unmarshal", "failed unmarshalling": protobuf unmarshal errors explained — this error's family across 10 libraries.

Related errors

• bad type: , want
• base64 decode for failed
• computeFacts: unable to check
• Could not unmarshal SourceConfig
• empty type

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/36f4df71ff65f2c2. Report an issue: GitHub.

Appendix: source

Thrown at sdks/go/pkg/beam/core/util/protox/any.go:82

// PackBase64Proto encodes a proto message into a base64-encoded BytesValue message.
func PackBase64Proto(in proto.Message) (*protobuf.Any, error) {
	data, err := EncodeBase64(in)
	if err != nil {
		return nil, err
	}
	return PackBytes([]byte(data))
}

// UnpackBytes removes the BytesValue wrapper.
func UnpackBytes(data *protobuf.Any) ([]byte, error) {
	if data.TypeUrl != bytesValueTypeURL {
		return nil, errors.Errorf("bad type: %v, want %v", data.TypeUrl, bytesValueTypeURL)
	}

	var buf protobufw.BytesValue
	if err := proto.Unmarshal(data.Value, &buf); err != nil {
		return nil, errors.Wrap(err, "BytesValue unmarshal failed")
	}
	return buf.Value, nil
}

// PackBytes applies a BytesValue wrapper to the supplied bytes.
func PackBytes(in []byte) (*protobuf.Any, error) {
	var buf protobufw.BytesValue
	buf.Value = in
	b, err := proto.Marshal(&buf)
	if err != nil {
		return nil, err
	}

	var out protobuf.Any
	out.TypeUrl = bytesValueTypeURL
	out.Value = b
	return &out, nil
}

View on GitHub (pinned to 12126d8942)