apache/beam · error

type must be bigtableio.Mutation but is: %v

Error message

type must be bigtableio.Mutation but is: %v

What it means

bigtableio requires that the PCollection being written consists of exactly bigtableio.Mutation elements. mustBeBigtableioMutation checks the element type at pipeline construction and rejects any other type with this error, failing fast before job submission.

Source

Thrown at sdks/go/pkg/beam/io/bigtableio/bigtable.go:124

	beam.ParDo0(s, &writeBatchFn{Project: project, InstanceID: instanceID, TableName: table, Type: beam.EncodedType{T: t}}, post)
}

func addGroupKeyFn(mutation Mutation) (int, Mutation) {
	if mutation.GroupKey != "" {
		return hashStringToInt(mutation.GroupKey), mutation
	}
	return 1, mutation
}

func hashStringToInt(s string) int {
	h := fnv.New32a()
	h.Write([]byte(s))
	return int(h.Sum32())
}

func mustBeBigtableioMutation(t reflect.Type) error {
	if t != reflect.TypeOf(Mutation{}) {
		return fmt.Errorf("type must be bigtableio.Mutation but is: %v", t)
	}
	return nil
}

type writeFn struct {
	// Project is the project
	Project string `json:"project"`
	// InstanceID is the bigtable instanceID
	InstanceID string `json:"instanceId"`
	// Client is the bigtable.Client
	client *bigtable.Client `json:"-"`
	// TableName is the qualified table identifier.
	TableName string `json:"tableName"`
	// Table is a bigtable.Table instance with an eventual open connection
	table *bigtable.Table `json:"-"`
	// Type is the encoded schema type.
	Type beam.EncodedType `json:"type"`
}

View on GitHub (pinned to 12126d8942)

Solutions

1. Map your data to bigtableio.Mutation values (with NewMutation / getBigtableMutation helpers or manual construction) before Write.
2. Ensure the PCollection is beam.V(Mutation{}) typed, not a pointer or slice.
3. Inspect the PCollection's element type at graph-construction time if types are inferred.

Example fix

// before
bigtableio.Write(s, project, instance, table, colFamily, typedCol)
// after
mutations := beam.ParDo(s, func(k string, v []byte) bigtableio.Mutation {
    return bigtableio.Mutation{RowKey: k, ColumnFamily: colFamily, Column: "data", Value: v}
}, typedCol)
bigtableio.Write(s, project, instance, table, colFamily, mutations)

if col.Type() != reflect.TypeOf(bigtableio.Mutation{}) {
    panic("PCollection element type must be bigtableio.Mutation")
}

func isMutationCol(col beam.Node) bool {
    return col.Type() == reflect.TypeOf(bigtableio.Mutation{})
}

defer func() {
    if r := recover(); r != nil && strings.Contains(fmt.Sprint(r), "type must be bigtableio.Mutation") {
        // fix PCollection element type before Write
    }
}()

When it happens

Trigger: Calling bigtableio.Write or bigtableio.WriteBatch with a PCollection whose element type is not bigtableio.Mutation (e.g. []byte, custom struct, Mutation pointer).

Common situations: Passing a custom row-mutation struct instead of converting to bigtableio.Mutation; using a pointer type *Mutation; feeding output of another transform directly without conversion.

Understand the failure class

Background: Type mismatch errors: IllegalArgumentException, TypeError and type guards across 150 open-source libraries — this error's family across 150 libraries.

Related errors

• could not create data operations client: %v
• invalid bigtableio.Mutation: %s
• invalid pos type: %T
• shardedKeyEncoder: Elm2 must be []byte shardID (got %T)
• unknown fieldtype: %T

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/e2598e70771d681f. Report an issue: GitHub.