apache/beam · error

bigqueryio.queryFn: failed to decode query parameters

Error message

bigqueryio.queryFn: failed to decode query parameters

What it means

Inside queryFn.ProcessElement (sdks/go/pkg/beam/io/bigqueryio/bigquery.go), the worker decodes the driver-encoded query parameters with decodeQueryParameters before issuing the BigQuery query; failure is wrapped as "bigqueryio.queryFn: failed to decode query parameters" and returned from the DoFn. It indicates the encoded parameter blob carried on the impulse side does not decode on the worker.

Solutions

1. Ensure driver and workers use the same Beam SDK version (rebuild/redeploy worker images).
2. Inspect the wrapped error to see whether it is a versioned format mismatch or corruption.
3. Re-submit the pipeline so parameters are freshly encoded with the current code.
4. Simplify parameters to supported scalar types to rule out type-specific decode failures.

Example fix

// before: mixed versions in deployment
go get github.com/apache/beam/sdks/go/pkg/beam@v2.40.0  # driver only
// after: pin same version everywhere
go get github.com/apache/beam/sdks/go/pkg/beam@v2.50.0
# and rebuild worker container image with the same version

// driver-side: confirm params round-trip
if _, err := decodeQueryParameters(encodeForTest(params)); err != nil {
    return fmt.Errorf("params will fail on worker: %w", err)
}

if err := doFnErr; strings.Contains(err.Error(), "failed to decode query parameters") {
    return fmt.Errorf("worker/driver Beam version mismatch: %w", err) // alert, resubmit
}

When it happens

Trigger: Running a pipeline where the worker's Beam version differs from the driver's (encoding/decoding format mismatch), or corrupt parameter bytes in the impulse side-input.

Common situations: Mixed SDK versions between job submission and workers (stale worker images), non-deterministic encoding of parameters, or custom modifications to encodeQueryParameters/decodeQueryParameters.

Understand the failure class

• Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

Related errors

• bigqueryio.Query: failed to encode query parameters
• bigqueryio: query parameter
• table name has empty components
• table name missing components
• A BigQuery table or a query must be specified

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/758181dcd62f56c5. Report an issue: GitHub.

Appendix: source

Thrown at sdks/go/pkg/beam/io/bigqueryio/bigquery.go:197

	QueryParameters []byte `json:"query_parameters"`
	// Options specifies additional query execution options.
	Options QueryOptions `json:"options"`
}

func (f *queryFn) ProcessElement(ctx context.Context, _ []byte, emit func(beam.X)) error {
	client, err := bigquery.NewClient(ctx, f.Project)
	if err != nil {
		return err
	}
	defer client.Close()

	q := client.Query(f.Query)
	if !f.Options.UseStandardSQL {
		q.UseLegacySQL = true
	}
	parameters, err := decodeQueryParameters(f.QueryParameters)
	if err != nil {
		return errors.Wrapf(err, "bigqueryio.queryFn: failed to decode query parameters")
	}
	q.Parameters = parameters

	it, err := q.Read(ctx)
	if err != nil {
		return err
	}

	for {
		val := reflect.New(f.Type.T).Interface() // val : *T
		if err := it.Next(val); err != nil {
			if err == iterator.Done {
				break
			}
			return err
		}

		emit(reflect.ValueOf(val).Elem().Interface()) // emit(*val)

View on GitHub (pinned to 12126d8942)