What causes "failed to matched a %v field for SQL column: %v" in apache/beam?

Add or update the struct field so its name (or case/underscore-insensitive variant) matches the SQL column exactly

failed to matched a %v field for SQL column: %v

Error message

failed to matched a %v field for SQL column: %v

What it means

databaseio maps SQL result columns to fields of your Go record struct via reflection (mapFields in util.go). It tries three matching strategies: exact name, case-insensitive name, and case-insensitive name with underscores stripped; an optional `column:"..."` struct tag is also consulted. When a returned column cannot be matched to any exported field of the struct, this error is thrown so the mismatch surfaces immediately instead of silently dropping data.

Source

Thrown at sdks/go/pkg/beam/io/databaseio/util.go:52

		fieldName := recordType.Field(i).Name
		indexedFields[fieldName] = i
		indexedFields[strings.ToLower(fieldName)] = i //to account for various matching strategies
		aTag := recordType.Field(i).Tag
		if column := aTag.Get("column"); column != "" {
			indexedFields[column] = i
		}
	}
	var mappedFieldIndex = make([]int, len(columns))
	for i, column := range columns {
		fieldIndex, ok := indexedFields[column]
		if !ok {
			fieldIndex, ok = indexedFields[strings.ToLower(column)]
		}
		if !ok {
			fieldIndex, ok = indexedFields[strings.Replace(strings.ToLower(column), "_", "", strings.Count(column, "_"))]
		}
		if !ok {
			return nil, errors.Errorf("failed to matched a %v field for SQL column: %v", recordType, column)
		}
		mappedFieldIndex[i] = fieldIndex
	}
	return mappedFieldIndex, nil
}

func asDereferenceSlice(aSlice []any) {
	for i, value := range aSlice {
		if value == nil {
			continue
		}
		aSlice[i] = reflect.ValueOf(value).Elem().Interface()

	}
}

func asMap(keys []string, values []any) map[string]any {
	var result = make(map[string]any)

View on GitHub (pinned to 12126d8942)

Solutions

Add or update the struct field so its name (or case/underscore-insensitive variant) matches the SQL column exactly
Add an explicit struct tag: `column:"the_sql_column_name"` on the matching field
Narrow the query to select only columns the struct actually has (avoid SELECT *)
Export the intended field — unexported fields are ignored by mapFields
Log/inspect the actual column list (e.g. via rows.Columns()) and compare to the struct fields

Example fix

// before
type User struct {
    id    int    // unexported: never matched
    Email string
}
// after
type User struct {
    ID    int    `column:"user_id"`
    Email string
}

Defensive patterns

Strategy: validation

Validate before calling

func validateColumnsMatchStruct(columns []string, t reflect.Type) error {
    for _, c := range columns {
        found := false
        for i := 0; i < t.NumField(); i++ {
            name := t.Field(i).Name
            tag := t.Field(i).Tag.Get("column")
            norm := strings.ReplaceAll(strings.ToLower(c), "_", "")
            if tag == c || name == c || strings.EqualFold(name, c) || strings.EqualFold(strings.ReplaceAll(name, "_", ""), norm) {
                found = true
                break
            }
        }
        if !found {
            return fmt.Errorf("no struct field for column %q", c)
        }
    }
    return nil
}

Prevention

Always declare explicit `column:"..."` tags on struct fields mapped to DB tables
Avoid SELECT *; list exact columns that match the struct
Keep DB schema and Go structs in sync; review migrations that add columns
Use exported fields only for mapped columns

When it happens

Trigger: Calling databaseio.Read (or Query) with a struct type whose exported fields do not cover all columns returned by the SQL query/statement — e.g. the query selects a column the struct lacks, the column name differs beyond the snake_case/case-insensitive heuristics, or the matching field is unexported and therefore skipped.

Common situations: Schema drift: a teammate added a DB column (or a SELECT *) but the Go struct wasn't updated; DBAs use naming conventions like camelCase-with-hyphens or quoted mixed-case identifiers the matcher can't normalize; forgetting the `column:"db_name"` tag when Go naming and SQL naming diverge; using an unexported field thinking it would still map.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/39430a79c3d5eea9. Report an issue: GitHub.