apache/beam · error

output type must be specified for sql.Transform

Error message

output type must be specified for sql.Transform

What it means

sql.Transform expands a SQL query into a cross-language pipeline segment; the expansion payload requires an output schema, supplied via an option. If no output type option was given, the constructor panics because it cannot describe the expansion result to the runner.

Source

Thrown at sdks/go/pkg/beam/transforms/sql/sql.go:108

// Example:
//
//	in := beam.Create(s, 1, 2, 3)
//	out := sql.Transform(s, "SELECT COUNT(*) FROM t",
//	    sql.Input("t", in),
//	    sql.OutputType(reflect.TypeOf(int64(0))))
//	// `out` is a PCollection<int64> with a single element 3.
//
// If an expansion service address is not provided as an option, one will be
// automatically started for the transform.
func Transform(s beam.Scope, query string, opts ...Option) beam.PCollection {
	o := &options{
		inputs: make(map[string]beam.PCollection),
	}
	for _, opt := range opts {
		opt(o)
	}
	if o.outType == nil {
		panic("output type must be specified for sql.Transform")
	}

	payload := beam.CrossLanguagePayload(&sqlx.ExpansionPayload{
		Query:   query,
		Dialect: o.dialect,
	})

	expansionAddr := sqlx.DefaultExpansionAddr
	if o.expansionAddr != "" {
		expansionAddr = xlangx.Require(o.expansionAddr)
	}

	out := beam.CrossLanguage(s, sqlx.Urn, payload, expansionAddr, o.inputs, beam.UnnamedOutput(o.outType))
	return out[graph.UnnamedOutputTag]
}

View on GitHub (pinned to 12126d8942)

Solutions

1. Pass sql.WithOutputType(beam.StructWith(...)) or the equivalent output-schema option to Transform.
2. Derive the output type from a registered Go struct matching the query's result schema.
3. Check that your option-building code path always appends the output-type option.
4. Refer to the transforms/sql package docs/tests for the exact option name and usage.

Example fix

// before
sql.Transform(s, "SELECT id, name FROM users")
// after
sql.Transform(s, "SELECT id, name FROM users", sql.WithOutputType(beam.StructWith(reflect.TypeOf(User{}))))

// Guard before calling Transform
func buildSQLTransform(s beam.Scope, query string, opts ...sql.Option) beam.PCollection {
	hasOut := false
	for _, o := range opts {
		if isOutputTypeOpt(o) {
			hasOut = true
		}
	}
	if !hasOut {
		opts = append(opts, sql.WithOutputType(beam.StructWith(reflect.TypeOf(Result{}))))
	}
	return sql.Transform(s, query, opts...)
}

When it happens

Trigger: Calling sql.Transform(scope, query) without the WithOutputType (output type) option, then building the pipeline — the check runs at graph-construction time in the calling main (or the test TestTransform_MissingOutputType).

Common situations: Porting examples where the option list is built conditionally, refactors that drop beamx/sql options, or users assuming the output schema is inferred from the query (it is not — it must be supplied client-side for cross-language expansion).

Understand the failure class

Background: "Must pass :limit option" / "Missing required option" — required option errors explained — this error's family across 41 libraries.

Related errors

• attempted to add namespace to missing coder id: %v not in %v
• attempted to add namespace to missing windowing strategy id:
• unable to partition query: %v
• res.GetError()
• columns were empty

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/639e9869d2b6bc0c. Report an issue: GitHub.