apache/beam · error

type has unexported field

Error message

type has unexported field: %v

What it means

While encoding a Go struct type for the pipeline wire format, encodeType rejects structs containing unexported fields (fields with non-empty PkgPath), because they cannot be reconstructed on the decoding side. This is a deliberate serialization-safety check in the Beam Go SDK's graph export path.

Solutions

1. Export all struct fields (capitalize field names) for types used in Beam signatures and PCollections.
2. Move unexported state (mutexes, caches) into separate non-serialized helper types outside the encoded struct.
3. Encode the private state as an exported field with a custom coder instead.
4. Use a distinct serializable struct for data flow and keep the encapsulating struct out of the graph.

Example fix

// before
type Event struct {
    TS   int64
    once sync.Once
}
// after
type Event struct {
    TS int64
}

func hasUnexportedFields(t reflect.Type) bool {
    if t.Kind() != reflect.Struct { return false }
    for i := 0; i < t.NumField(); i++ {
        if t.Field(i).PkgPath != "" { return true }
    }
    return false
}

func beamSafeStruct(v any) bool { return !hasUnexportedFields(reflect.TypeOf(v)) }

if hasUnexportedFields(reflect.TypeOf(evt)) {
    return fmt.Errorf("type %T has unexported fields and cannot be serialized by Beam", evt)
}

When it happens

Trigger: Any pipeline serialization that encounters a struct type with at least one lowercase (unexported) field: DoFn structs, PCollection element types, or fields nested inside function signatures, during graphx export.

Common situations: User DoFns or event types modeled with unexported fields (idiomatic Go encapsulation, e.g. mutexes sync.Mutex or private caches) submitted to remote runners.

Related errors

• bad field type
• bad base type
• bad channel direction
• bad element type
• bad parameter type

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/1ead3193597e644e. Report an issue: GitHub.

Appendix: source

Thrown at sdks/go/pkg/beam/core/runtime/graphx/serialize.go:442

		return &v1pb.Type{Kind: v1pb.Type_FLOAT64}, nil
	case reflect.String:
		return &v1pb.Type{Kind: v1pb.Type_STRING}, nil

	case reflect.Slice:
		elm, err := encodeType(t.Elem())
		if err != nil {
			wrapped := errors.Wrap(err, "bad element type")
			return nil, errors.WithContextf(wrapped, "encoding slice %v", t)
		}
		return &v1pb.Type{Kind: v1pb.Type_SLICE, Element: elm}, nil

	case reflect.Struct:
		var fields []*v1pb.Type_StructField
		for i := 0; i < t.NumField(); i++ {
			f := t.Field(i)

			if f.PkgPath != "" {
				wrapped := errors.Errorf("type has unexported field: %v", f.Name)
				return nil, errors.WithContextf(wrapped, "encoding struct %v", t)
			}

			fType, err := encodeType(f.Type)
			if err != nil {
				wrapped := errors.Wrap(err, "bad field type")
				return nil, errors.WithContextf(wrapped, "encoding struct %v", t)
			}

			field := &v1pb.Type_StructField{
				Name:      f.Name,
				PkgPath:   f.PkgPath,
				Type:      fType,
				Tag:       string(f.Tag),
				Offset:    int64(f.Offset),
				Index:     encodeInts(f.Index),
				Anonymous: f.Anonymous,
			}

View on GitHub (pinned to 12126d8942)