What causes "columns were empty" in apache/beam?

Pass an explicit non-empty column list to databaseio.Write, e.g. []string{"id", "name"}.

columns were empty

Error message

columns were empty

What it means

databaseio's newWriter builds SQL INSERT statements from a column list; an empty columns slice would produce malformed SQL, so it rejects the call upfront with 'columns were empty'. The writer cannot determine the table's column set without it.

Source

Thrown at sdks/go/pkg/beam/io/databaseio/writer.go:93

}

func (w *writer) writeBatchIfNeeded(ctx context.Context, db *sql.DB) error {
	if w.rowCount >= w.batchSize {
		return w.write(ctx, db)
	}
	return nil
}

func (w *writer) writeIfNeeded(ctx context.Context, db *sql.DB) error {
	if w.rowCount >= 0 {
		return w.write(ctx, db)
	}
	return nil
}

func newWriter(driver string, batchSize int, table string, columns []string) (*writer, error) {
	if len(columns) == 0 {
		return nil, errors.New("columns were empty")
	}
	return &writer{
		batchSize:              batchSize,
		columnCount:            len(columns),
		table:                  table,
		binding:                make([]any, 0),
		sqlTemplate:            fmt.Sprintf("INSERT INTO %v(%v) VALUES", table, strings.Join(columns, ",")),
		valueTemplateGenerator: &valueTemplateGenerator{driver},
	}, nil
}

type valueTemplateGenerator struct {
	driver string
}

func (v *valueTemplateGenerator) generate(rowCount int, columnColunt int) string {
	switch v.driver {
	case "postgres", "pgx":

View on GitHub (pinned to 12126d8942)

Solutions

Pass an explicit non-empty column list to databaseio.Write, e.g. []string{"id", "name"}.
If columns are generated from a schema, verify the schema file/data is loaded and parsed before constructing the write transform.
Validate len(columns) > 0 in your pipeline-construction code and fail with a descriptive message.

Example fix

// before
databaseio.Write(scope, db, "mytable", nil, reflect.TypeOf(row)) // columns empty

// after
cols := []string{"id", "name", "created_at"}
databaseio.Write(scope, db, "mytable", cols, reflect.TypeOf(row))

Defensive patterns

Strategy: validation

Validate before calling

if len(columns) == 0 {
	return fmt.Errorf("databaseio.Write requires a non-empty column list")
}
// proceed with databaseio.Write(..., columns, ...)

Prevention

Explicitly list table columns rather than deriving them from possibly-empty sources.
Validate schema/config files load and contain column definitions before pipeline construction.
Fail fast at startup when the column list is empty.

When it happens

Trigger: Writing to a database via databaseio.Write (which routes through ProcessElement -> newWriter) with an empty columns parameter — e.g. columns derived from a schema introspection call that returned nothing, or a config key that was missing/empty.

Common situations: Auto-deriving columns from an empty Avro/JSON schema file; passing a nil slice from an env/config variable that was never populated; a table-name typo causing reflection/introspection to yield no columns.

Understand the failure class

Background: "must not be empty", "cannot be empty" — required-field validation errors across open-source libraries — this error's family across 41 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/c1996691c36ee190. Report an issue: GitHub.