{"record":{"id":"e4bf5dad29ecf5d0","repo":"vitessio/vitess","slug":"stream-send-error-v-e4bf5d","errorCode":null,"errorMessage":"stream send error: %v","messagePattern":"stream send error: (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"go/vt/vttablet/tabletserver/vstreamer/resultstreamer.go","lineNumber":98,"sourceCode":"\tif rotatedLog {\n\t\trs.vse.vstreamerFlushedBinlogs.Add(1)\n\t}\n\tif err != nil {\n\t\treturn err\n\t}\n\n\t// first call the callback with the fields\n\tflds, err := conn.Fields()\n\tif err != nil {\n\t\treturn err\n\t}\n\n\terr = rs.send(&binlogdatapb.VStreamResultsResponse{\n\t\tFields: flds,\n\t\tGtid:   gtid,\n\t})\n\tif err != nil {\n\t\treturn fmt.Errorf(\"stream send error: %v\", err)\n\t}\n\n\tresponse := &binlogdatapb.VStreamResultsResponse{}\n\tbyteCount := 0\n\tloggerName := fmt.Sprintf(\"%s (%v)\", rs.vse.GetTabletInfo(), rs.tableName)\n\tlogger := logutil.NewThrottledLogger(loggerName, throttledLoggerInterval)\n\tfor {\n\t\tselect {\n\t\tcase <-rs.ctx.Done():\n\t\t\treturn fmt.Errorf(\"stream ended: %v\", rs.ctx.Err())\n\t\tdefault:\n\t\t}\n\n\t\t// check throttler.\n\t\tif _, ok := rs.vse.throttlerClient.ThrottleCheckOKOrWaitAppName(rs.ctx, throttlerapp.ResultStreamerName); !ok {\n\t\t\tlogger.Infof(\"throttled.\")\n\t\t\tcontinue\n\t\t}","sourceCodeStart":80,"sourceCodeEnd":116,"githubUrl":"https://github.com/vitessio/vitess/blob/01a25a7d176f94613b8d59d799f438380a8760e4/go/vt/vttablet/tabletserver/vstreamer/resultstreamer.go#L80-L116","documentation":"During VStreamResults, resultstreamer.Stream sends the initial fields/Gtid response over the VStreamResults gRPC stream. If the send callback fails (client disconnected, context canceled, RPC broken), it is wrapped as 'stream send error: %v'. The stream cannot continue without a live consumer, so the error propagates back to StreamResults.","triggerScenarios":"The VStreamResults client closes its connection or cancels its context while the tablet is streaming results; network interruption between vtctld/vtgate and the tablet mid-stream; the send callback returns a transport error on the initial Fields/Gtid response or during row sends.","commonSituations":"Consumer timeout or restart while streaming a large table snapshot; load balancer killing long-lived streams; tablet draining/restart during a VStreamResults call; client crash during MoveTables-style result streaming.","solutions":["Inspect the wrapped cause for the actual transport error (EOF, context canceled, connection refused).","Restart the VStreamResults request from a known gtid once the consumer is healthy.","Check connectivity/timeout settings between the consumer and the tablet (gRPC keepalives, LB idle timeouts).","Retry with backoff — streams are resumable by design via the gtid; treat this as a transient stream failure unless the cause is deterministic."],"exampleFix":"// before\nresp, err := vstreamResults(ctx, tablet, query) // one-shot, dies on transient send error\n// after\nfor attempt := 0; attempt < maxRetries; attempt++ {\n    resp, err := vstreamResults(ctx, tablet, query)\n    if err == nil || !isTransientStreamError(err) {\n        break\n    }\n    time.Sleep(backoff(attempt))\n}","handlingStrategy":"retry","validationCode":"// before streaming, confirm the tablet is reachable and the stream target is serving\nif err := tabletHealthCheck(ctx, tabletAlias); err != nil {\n    return fmt.Errorf(\"tablet not ready for VStreamResults: %w\", err)\n}","typeGuard":"func isTransientStreamError(err error) bool {\n    s := err.Error()\n    return strings.Contains(s, \"stream send error\") &&\n        (strings.Contains(s, \"context canceled\") || strings.Contains(s, \"EOF\") ||\n            strings.Contains(s, \"Unavailable\") || strings.Contains(s, \"Broken pipe\"))\n}","tryCatchPattern":"err := streamVStreamResults(ctx, tablet, query, handler)\nif err != nil {\n    if isTransientStreamError(err) {\n        return retryWithBackoff(ctx, func() error { return streamVStreamResults(ctx, tablet, query, handler) })\n    }\n    return fmt.Errorf(\"VStreamResults failed permanently: %w\", err)\n}","preventionTips":["Resume streams from the last received gtid instead of full re-snapshots","Set gRPC keepalives and LB idle timeouts longer than expected stream duration","Handle consumer restarts gracefully — treat send errors as resumable","Avoid streaming huge snapshots without checkpointing progress"],"tags":["vstreamer","grpc","streaming","network"],"backgroundTag":"stream-send-error","analyzedSha":"01a25a7d176f94613b8d59d799f438380a8760e4","analyzedAt":"2026-09-01T17:28:30.605Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}