apache/beam · error

unable to do row encoding

Error message

unable to do row encoding

What it means

In jdbcio.toRow, after successfully obtaining a row encoder, the actual encoding step (enc(pl, &buf)) can still fail; in that case the function panics with 'unable to do row encoding'. This means the value's runtime shape failed to encode into a Beam row buffer even though a matching encoder was found.

Solutions

1. Filter out nil elements before the JDBC write/read transform so enc never receives a nil value.
2. Keep element types consistent: encode either all values or all pointers of the struct, matching the type used when the encoder was created.
3. Rebuild/redeploy the pipeline after any struct field changes so encoders match the new schema.
4. If the panic persists, inspect the dropped error by patching toRow locally (log err from enc) to identify the offending field.

Example fix

// before
p | beam.Create(ctx, []*User{nil, {ID: 1}}) | jdbcio.Write(...) // panic: unable to do row encoding

// after
filtered := beam.Filter(p, func(u *User) bool { return u != nil },
    beam.Create(p, []*User{nil, {ID: 1}}))
filtered | jdbcio.Write(s, jdbcio.IO{...})

func validUser(u *User) bool { return u != nil }
filtered := beam.Filter(s, validUser, input)

func nonNil[T any](v *T) bool { return v != nil }

When it happens

Trigger: Calling jdbcio.Read/Write where a struct element matches the registered encoder but the concrete value passed to enc does not conform — e.g. a nil pointer, a value whose dynamic type differs from the type the encoder was built for, or an internal encoding failure on a field value.

Common situations: Passing a nil *Struct to the transform; mixing value and pointer elements of the same struct type in one PCollection so the encoder's expected type doesn't match the runtime value; schema mismatch after changing struct fields without rebuilding.

Understand the failure class

Background: json.Marshal / "failed to marshal" errors in Go: why "unsupported type" happens and how to fix it — this error's family across 22 libraries.

Related errors

• unable to get row encoder
• Cannot encode payload for WriteToPubSub. Expected valid…
• cannot make a keyed iterable for an unkeyed side input
• cannot make a state provider for an unkeyed input
• err

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/31fc3dee0f6c4c5f. Report an issue: GitHub.

Appendix: source

Thrown at sdks/go/pkg/beam/io/xlang/jdbcio/jdbc.go:113

// jdbcConfig stores the expansion service and configuration for JDBC IO.
type jdbcConfig struct {
	classpaths    []string
	expansionAddr string
	config        *config
}

// TODO(riteshghorse): update the IO to use wrapper created in BigQueryIO.
func toRow(pl any) []byte {
	rt := reflect.TypeOf(pl)

	enc, err := coder.RowEncoderForStruct(rt)
	if err != nil {
		panic(fmt.Errorf("unable to get row encoder"))
	}
	var buf bytes.Buffer
	if err := enc(pl, &buf); err != nil {
		panic(fmt.Errorf("unable to do row encoding"))
	}
	return buf.Bytes()
}

// Write is a cross-language PTransform which writes Rows to the specified database via JDBC.
// tableName is a required parameter, and by default, the write statement is generated from it.
// The generated write statement can be overridden by passing in a WriteStatement option.
// If an expansion service address is not provided,
// an appropriate expansion service will be automatically started; however
// this is slower than having a persistent expansion service running.
//
// If no additional classpaths are provided using jdbcio.WriteClasspaths() then the default classpath
// for that driver would be used. As of now, the default classpaths are present only for PostgreSQL and MySQL.
//
// The default write statement is: "INSERT INTO tableName(column1, ...) INTO VALUES(value1, ...)"
// Example:
//
//	  tableName := "roles"

View on GitHub (pinned to 12126d8942)