apache/beam · critical

schema type must be struct

Error message

schema type must be struct: %v

What it means

mustInferSchema in bigqueryio panics when the Go type whose schema is being inferred for BigQuery IO is not a struct. bigquery.InferSchema only works on struct types (fields map to BigQuery columns), so any other kind (int, slice, map, pointer-to-non-struct, etc.) is rejected immediately. This is a programming-time contract violation, not a runtime data problem.

Solutions

1. Define a named struct whose exported fields represent the BigQuery row and make the PCollection emit that struct.
2. Ensure the PCollection passed to Read/Write has element type a struct (check col.Type().Type().Kind() == reflect.Struct before calling).
3. If the data is not row-shaped, insert a beam.ParDo that wraps each element into the row struct before writing.

Example fix

// before
bigqueryio.Write(s, proj, ds, table, beam.Create(s, "a", "b"))

// after
type Row struct {
    Name string `bigquery:"name"`
}
bigqueryio.Write(s, proj, ds, table, beam.Create(s, Row{Name: "a"}, Row{Name: "b"}))

if col.Type().Type().Kind() != reflect.Struct {
    return fmt.Errorf("bigqueryio requires a struct element type, got %v", col.Type().Type())
}

func isStructElem(col beam.PCollection) bool {
    return col.Type().Type().Kind() == reflect.Struct
}

func safeWrite(s beam.Scope, args ...) {
    defer func() {
        if r := recover(); r != nil {
            log.Fatalf("bigquery write setup failed: %v", r)
        }
    }()
    bigqueryio.Write(s, args...)
}

When it happens

Trigger: Calling bigqueryio.Read(s, project, dataset, table, beam.Create(s, 42)) or bigqueryio.Write with a PCollection whose element type is not a named struct — e.g. a PCollection<string>, PCollection<int>, or a slice/map element type.

Common situations: Feeding a plain scalar PCollection (from TextIO or Create of literals) into bigquery.Write; passing a []byte or map element; accidentally dereferencing to a non-struct type; writing a pipeline where an upstream DoFn emits primitives instead of row structs.

Understand the failure class

Background: Type mismatch errors: IllegalArgumentException, TypeError and type guards across 150 open-source libraries — this error's family across 150 libraries.

Related errors

• err
• Nested FullValues must be nested as pointers.
• pcollection must be of KV type
• pcollection must be of non-composite type
• pubsubio.Write only accepts PCollections of

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/2916f81ee3e5d63b. Report an issue: GitHub.

Appendix: source

Thrown at sdks/go/pkg/beam/io/bigqueryio/bigquery.go:222

	}

	for {
		val := reflect.New(f.Type.T).Interface() // val : *T
		if err := it.Next(val); err != nil {
			if err == iterator.Done {
				break
			}
			return err
		}

		emit(reflect.ValueOf(val).Elem().Interface()) // emit(*val)
	}
	return nil
}

func mustInferSchema(t reflect.Type) bigquery.Schema {
	if t.Kind() != reflect.Struct {
		panic(fmt.Sprintf("schema type must be struct: %v", t))
	}

	checkTypeRegistered(t)

	schema, err := bigquery.InferSchema(reflect.Zero(t).Interface())
	if err != nil {
		panic(errors.Wrapf(err, "invalid schema type: %v", t))
	}
	return schema
}

func checkTypeRegistered(t reflect.Type) {
	t = reflectx.SkipPtr(t)
	key, ok := runtime.TypeKey(t)
	if !ok {
		panic(fmt.Sprintf("type %v must be a named type (not anonymous) for registration", t))
	}

View on GitHub (pinned to 12126d8942)