vitessio/vitess · error

did not received expected fields in response %+v on tablet %

Error message

did not received expected fields in response %+v on tablet %v

What it means

In the differ's VStreamRows loop, the first response must carry field metadata; if the first streamed response has neither cached fields nor vsr.Fields, the differ cannot decode subsequent rows and returns this error with the offending response and tablet alias. (The typo 'did not received' is in the source.)

Source

Thrown at go/vt/vttablet/tabletmanager/vdiff/table_differ.go:466

		}
		var fields []*querypb.Field
		req := &binlogdatapb.VStreamRowsRequest{
			// We pass the NoTimeouts options as otherwise the row streamer will add a MAX_EXECUTION_TIME
			// query hint with a value based on the --vreplication-copy-phase-duration flag.
			Target: target, Query: query, Lastpk: lastPK, Options: &binlogdatapb.VStreamOptions{NoTimeouts: true},
		}
		return conn.VStreamRows(ctx, req, func(vsrRaw *binlogdatapb.VStreamRowsResponse) error {
			// We clone (deep copy) the VStreamRowsResponse -- which contains a vstream packet with N rows and
			// their corresponding GTID position/snapshot along with the LastPK in the row set -- so that we
			// can safely process it while the next VStreamRowsResponse message is getting prepared by the
			// shardStreamer. Without doing this, we would have to serialize the row processing by using
			// unbuffered channels which would present a major performance bottleneck.
			// This need arises from the gRPC VStreamRowsResponse pooling and re-use/recycling done for
			// gRPCQueryClient.VStreamRows() in vttablet/grpctabletconn/conn.
			vsr := vsrRaw.CloneVT()
			if len(fields) == 0 {
				if len(vsr.Fields) == 0 {
					return fmt.Errorf("did not received expected fields in response %+v on tablet %v",
						vsr, td.wd.ct.vde.thisTablet.Alias)
				}
				fields = vsr.Fields
				gtidch <- vsr.Gtid
			}
			if len(vsr.Rows) == 0 && len(vsr.Fields) == 0 {
				return nil
			}
			p3qr := &querypb.QueryResult{
				Fields: fields,
				Rows:   vsr.Rows,
			}
			result := sqltypes.Proto3ToResult(p3qr)

			// Fields should be received only once, and sent only once.
			if len(vsr.Fields) == 0 {
				result.Fields = nil
			}

View on GitHub (pinned to 01a25a7d17)

Solutions

  1. Retry the vdiff; transient RPC glitches often resolve on a fresh stream.
  2. Check tablet logs around the time for VStreamRows errors and version skew between vtctld/vttablet.
  3. Upgrade to a consistent Vitess version across components; if reproducible, file an issue with the tablet logs.
Defensive patterns

Strategy: retry

Try / catch

err := runTableDiff()
if err != nil && strings.Contains(err.Error(), "did not received expected fields") {
  // transient stream/protocol issue; retry once with backoff
  time.Sleep(2 * time.Second)
  err = runTableDiff()
}

Prevention

When it happens

Trigger: The first VStreamRows response from the target arrives with empty Fields and no previously cached fields — e.g. a protocol/RPC anomaly, response pooling bug, or the stream erroring right at start.

Common situations: gRPC connection issues between target differ and vttablet; mixed-version cluster where response reuse/recycling misbehaves; target tablet restarted mid-stream so fields never arrive.

Related errors


AI-assisted analysis of vitessio/vitess@01a25a7d17 (2026-09-01). Data as JSON: /api/errors/8340babada3522b0. Report an issue: GitHub.