{"record":{"id":"04fd68f75b01c214","repo":"vitessio/vitess","slug":"uncaught-panic-v-vtgate-v","errorCode":null,"errorMessage":"uncaught panic: %v, vtgate: %v","messagePattern":"uncaught panic: (.+?), vtgate: (.+?)","errorType":"panic","errorClass":null,"httpStatus":null,"severity":"critical","filePath":"go/vt/vtgate/vtgate.go","lineNumber":1027,"sourceCode":"\t\t}\n\t\t// log only if sql query present and able to successfully redact the PII.\n\t\tlogger.Infof(\"unsupported query: %q\", piiSafeSQL)\n\t}\n\treturn err\n}\n\nfunc formatError(err error) error {\n\tif err == nil {\n\t\treturn nil\n\t}\n\treturn err\n}\n\n// HandlePanic recovers from panics, and logs / increment counters\nfunc (vtg *VTGate) HandlePanic(err *error) {\n\tif x := recover(); x != nil {\n\t\tlog.Error(fmt.Sprintf(\"Uncaught panic:\\n%v\\n%s\", x, tb.Stack(4)))\n\t\t*err = fmt.Errorf(\"uncaught panic: %v, vtgate: %v\", x, servenv.ListeningURL.String())\n\t\terrorCounts.Add([]string{\"Panic\", \"Unknown\", \"Unknown\", vtrpcpb.Code_INTERNAL.String()}, 1)\n\t}\n}\n\nfunc newVTGate(executor *Executor, resolver *Resolver, vsm *vstreamManager, tc *TxConn, gw *TabletGateway) *VTGate {\n\treturn &VTGate{\n\t\texecutor:                executor,\n\t\tresolver:                resolver,\n\t\tvsm:                     vsm,\n\t\ttxConn:                  tc,\n\t\tgw:                      gw,\n\t\ttimings:                 timings,\n\t\trowsReturned:            rowsReturned,\n\t\trowsAffected:            rowsAffected,\n\t\tqueryTextCharsProcessed: queryTextCharsProcessed,\n\n\t\tlogExecute:       logutil.NewThrottledLogger(\"Execute\", 5*time.Second),\n\t\tlogPrepare:       logutil.NewThrottledLogger(\"Prepare\", 5*time.Second),","sourceCodeStart":1009,"sourceCodeEnd":1045,"githubUrl":"https://github.com/vitessio/vitess/blob/01a25a7d176f94613b8d59d799f438380a8760e4/go/vt/vtgate/vtgate.go#L1009-L1045","documentation":"VTGate recovered from an unrecovered panic inside a request-handling goroutine. The panic value is re-thrown to the caller as an error with the vtgate listening URL appended so the client knows which gate instance handled (and crashed on) the request. A 'Panic' counter is also incremented with code INTERNAL.","triggerScenarios":"Any panic during query execution in vtgate (nil pointer in executor, malformed internal state, bug in a plugin/resolver) that is caught by the deferred HandlePanic(err) in the RPC handler wrappers.","commonSituations":"Hitting an untested code path in a query plan, a bug after a Vitess upgrade, corrupt session state, or concurrency bugs surfacing as nil dereferences during load.","solutions":["Read the full 'Uncaught panic:' stack trace in the vtgate log at the reported URL to identify the panicking function","Check the panic value: nil-pointer on a nil session/plan usually indicates a client sending requests with unexpected fields","Search Vitess issues for the panicking function; upgrade to a release containing the fix","If reproducible, file a bug with the stack trace, query, and schema/vschema","As mitigation, retry against a different vtgate instance since the panic is contained per-request"],"exampleFix":"// before: opaque INTERNAL error surfaced to client\n// after: capture root cause from logs and avoid trigger\n// log: Uncaught panic: runtime error: invalid memory address ... vtgate/exec.go:123\n// action: upgrade Vitess to version fixing exec.go:123, or stop sending the offending query shape","handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"res, err := vtgateConn.Execute(ctx, target, sql, bindVars)\nif err != nil && strings.Contains(err.Error(), \"uncaught panic\") {\n    // request killed vtgate internals; capture err (contains vtgate URL),\n    // retry on another vtgate or report with the stack trace from server logs\n    return nil, fmt.Errorf(\"vtgate panicked (%w); see server logs at reported URL\", err)\n}","preventionTips":["Keep vtgate on the latest patch release — panics are fixed as INTERNAL bugs","Monitor the errorCounts 'Panic' counter and alert on any increase","Correlate the panic's vtgate URL in the error with server-side stack traces","Avoid untested query shapes/features against production vtgate until validated in staging"],"tags":["panic","vtgate","internal-error","recovery"],"backgroundTag":"uncaught-panic-recovered","analyzedSha":"01a25a7d176f94613b8d59d799f438380a8760e4","analyzedAt":"2026-09-01T17:28:30.605Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}